Binning and non-binning combination

ABSTRACT

A LIDAR system may include a processor configured to control a LIDAR light source for illuminating a field of view (FOV), receive, from a group of light detectors, input signals indicative of reflections of light from objects in the FOV, and process a first subset of the input signals associated with a first region of the FOV to detect a first object in the first region. The processor may also process a second subset of the input signals associated with a second region of the FOV to detect a second object in the second region. The second object may be located at a greater distance from the light source than the first object.

CROSS REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of priority of U.S. ProvisionalPatent Application No. 62/563,367, filed Sep. 26, 2017; U.S. ProvisionalPatent Application No. 62/567,692, filed Oct. 3, 2017; U.S. ProvisionalPatent Application No. 62/589,686, filed Nov. 22, 2017; and U.S.Provisional Patent Application No. 62/632,789, filed Feb. 20, 2018. Allof the foregoing applications are incorporated herein by reference intheir entirety.

BACKGROUND I. Technical Field

The present disclosure relates generally to surveying technology forscanning a surrounding environment, and, more specifically, to systemsand methods that use LIDAR technology to detect objects in thesurrounding environment.

II. Background Information

With the advent of driver assist systems and autonomous vehicles,automobiles need to be equipped with systems capable of reliably sensingand interpreting their surroundings, including identifying obstacles,hazards, objects, and other physical parameters that might impactnavigation of the vehicle. To this end, a number of differingtechnologies have been suggested including radar, LIDAR, camera-basedsystems, operating alone or in a redundant manner.

One consideration with driver assistance systems and autonomous vehiclesis an ability of the system to determine surroundings across differentconditions including, rain, fog, darkness, bright light, and snow. Alight detection and ranging system, (LIDAR a/k/a LADAR) is an example oftechnology that can work well in differing conditions, by measuringdistances to objects by illuminating objects with light and measuringthe reflected pulses with a sensor. A laser is one example of a lightsource that can be used in a LIDAR system. As with any sensing system,in order for a LIDAR-based sensing system to be fully adopted by theautomotive industry, the system should provide reliable data enablingdetection of far-away objects. Currently, however, the maximumillumination power of LIDAR systems is limited by the need to make theLIDAR systems eye-safe (i.e., so that they will not damage the human eyewhich can occur when a projected light emission is absorbed in the eye'scornea and lens, causing thermal damage to the retina.)

The systems and methods of the present disclosure are directed towardsimproving performance of LIDAR systems while complying with eye safetyregulations.

SUMMARY

In some embodiments, a LIDAR system may include at least one processorconfigured to control at least one light source for projecting lighttoward a field of view and receive from at least one first sensor firstsignals associated with light projected by the at least one light sourceand reflected from an object in the field of view, wherein the lightimpinging on the at least one first sensor is in a form of a light spothaving an outer boundary. The processor may further be configured toreceive from at least one second sensor second signals associated withlight noise, wherein the at least one second sensor is located outsidethe outer boundary; determine, based on the second signals received fromthe at least one second sensor, an indicator of a magnitude of the lightnoise; and determine, based on the indicator the first signals receivedfrom the at least one first sensor and, a distance to the object.

Some embodiments may also include a method for using LIDAR to detectobjects. The method may include controlling at least one light sourcefor projecting light toward a field of view; receiving from at least onefirst sensor signals associated with light projected by the at least onelight source and reflected from an object in the field of view, whereinlight impinging on the at least one first sensor is in a form of a lightspot having an outer boundary; receiving from at least one secondsensor, signals associated with light noise, wherein the at least onesecond sensor is located outside the outer boundary; determining, basedon the signals received from the at least one second sensor, anindicator of a magnitude of the light noise; and determining, based onthe signals received from the at least one first sensor and theindicator, a distance to the object.

Additionally, some embodiments may include a non-transitorycomputer-readable storage medium including instructions that, whenexecuted by at least one processor, cause the at least one processor toperform a method for compensating for light noise in a LIDAR system. Themethod may include receiving from at least one first sensor signalsassociated with light projected by at least one light source andreflected from an object, wherein light impinging on the at least onefirst sensor is in a form of a light spot having an outer boundary;receiving from at least one second sensor, signals associated with lightnoise, wherein the at least one second sensor is located outside theouter boundary; determining, based on the signals received from the atleast one second sensor, an indicator of a magnitude of the light noise;and correcting the signals received from the at least one first sensorusing the indicator to compensate for the light noise, thereby enablinga determination of a distance to the object.

In an exemplary embodiment, a LIDAR system for projecting light througha protective window associated with the LIDAR system may include atleast one processor. The at least one processor may be configured to:control at least one LIDAR light source; receive LIDAR reflectionssignals from at least one sensor, wherein the LIDAR reflections signalsinclude indications of light reflected from the protective window andlight reflected from objects in the field of view and passing throughthe protective window prior to reaching the at least one sensor; detect,based on the LIDAR reflections signals, a particular obstruction patternat least partially obstructing light passage through the protectivewindow; access stored information characterizing reference obstructionpatterns for at least one of salt, mud, road grime, snow, rain, dust,bug debris, pollen, and bird droppings; compare the detected obstructionpattern with the reference obstruction patterns in order to determine alikely obstruction-pattern match; and based on the likely match, outputinformation indicative of the match.

In another exemplary embodiment, a LIDAR system for projecting lightthrough a protective window associated with the LIDAR system may includeat least one processor. The at least one processor may be configured to:control at least one LIDAR light source; receive LIDAR reflectionssignals from at least one sensor, wherein the LIDAR reflections signalsinclude indications of light reflected from objects in the field of viewand passing through the protective window prior to reaching the at leastone sensor; detect, based on the LIDAR reflections signals, a particularobstruction area at least partially obstructing light passage throughthe protective window at a first time; and initiate at least oneremedial action, based on the detected obstruction area, to increase theamount of light passaging through the protective at a second time.

In another exemplary embodiment, a vehicle may include a body; a LIDARlight source arranged to projecting light through a protective window;and at least one processor. The at least one processor may be configuredto control the at least one LIDAR light source; receive LIDARreflections signals from at least one sensor, wherein the LIDARreflections signals are indicative of light reflected from objects in afield of view and passing through the protective window prior toreaching the at least one sensor; detect, based on the LIDARreflections, a particular obstruction pattern at least partiallyobstructing light passage through the protective window; access storedinformation characterizing reference obstruction patterns for at leastone of salt, mud, road grime, snow, rain, dust, bug debris, pollen, andbird droppings; compare the detected obstruction pattern with thereference obstruction patterns in order to determine a likely match; andbased on the likely match, output information indicative of the match.

In another exemplary embodiment, a method for determining obstructionson a protective window associated with a LIDAR may include: controllingat least one LIDAR light source; receiving LIDAR reflections signalsfrom at least one sensor, wherein the LIDAR reflections signals areindicative of light reflected from objects in a field of view andpassing through the protective window prior to reaching the at least onesensor; detecting, based on the LIDAR reflections, a particularobstruction pattern at least partially obstructing light passage throughthe protective window; accessing stored information characterizingreference obstruction patterns for at least one of salt, mud, roadgrime, snow, rain, dust, bug debris, pollen, and bird droppings;comparing the detected obstruction pattern with the referenceobstruction patterns in order to determine a likely match; and based onthe likely match, output information indicative of the match.

In another exemplary embodiment, a LIDAR system for projecting lightthrough a protective window associated with the LIDAR system may includeat least one processor. The at least one processor may be configured to:control at least one LIDAR light source; receive LIDAR reflectionssignals from at least one sensor, wherein the LIDAR reflections signalsinclude indications of light reflected from the protective window insidethe LIDAR system; determine internal reflection parameters from theLIDAR reflections signals; access memory storing signal baselineparameters associated with the LIDAR system; use the internal reflectionparameters and the signal baseline parameters to identify at least oneobstructed portion of the field of view; and alter a light sourceparameter such that more light is projected toward other portions of thefield of view than light projected toward the at least one obstructedportion of the field of view.

In one embodiment, a LIDAR system may comprise at least one processor.The at least one processor may be configured to control at least oneLIDAR light source in a manner enabling light flux to vary over aplurality of scans of a field of view. The field of view may include aforeground area and a background area. The at least one processor may befurther configured to receive from at least one detector a plurality ofinput signals indicative of light reflected from the field of view. Arepresentation of a portion of the field of view associated with aplurality of pixels may be constructible from the plurality of inputsignals, and the plurality of input signals may be associated with afirst pixel that covers a portion of the foreground area, a second pixelthat covers a portion of the foreground area and a portion of thebackground area, and a third pixel that covers a portion of thebackground area. The at least one processor may be further configured touse input signals associated with the first pixel to determine adistance to a first object located in the foreground area and use inputsignals associated with the second pixel and input signals associatedwith the third pixel to determine a distance to a second object locatedin the background area.

In one embodiment, a vehicle may comprise a body and at least oneprocessor within the body. The at least one processor may be configuredto control at least one LIDAR light source in a manner enabling lightprojected from the at least one LIDAR light source to vary over aplurality of scans of a field of view. The field of view may include aforeground area and a background area. The at least one processor may befurther configured to receive, from a group of detectors, a plurality ofinput signals indicative of reflections of the projected light from thefield of view. A representation of a portion of the field of viewassociated with a plurality of pixels may be constructible from theplurality of input signals, and the plurality of input signals may beassociated with a first pixel that covers a portion of the foregroundarea, a second pixel that covers a portion of the foreground area and aportion of the background area, and a third pixel that covers a portionof the background area. The at least one processor may be furtherconfigured to use input signals associated with the first pixel todetermine a distance to a first object located in the foreground areaand use input signals associated with the second pixel and input signalsassociated with the third pixel to determine a distance to a secondobject located in the background area.

In one embodiment, a method for using LIDAR to determine distances toobjects in a field of view may comprise controlling at least one LIDARlight source in a manner enabling light projected from the at least onelight source to vary over a plurality of scans of a field of view. Thefield of view may include a foreground area and a background area. Themethod may further comprise receiving from a group of detectors aplurality of input signals indicative of reflections of the projectedlight from the field of view. A representation of a portion of the fieldof view associated with a plurality of pixels may be constructible fromthe plurality of input signals. The method may further compriseidentifying a first pixel that covers a portion of the foreground area,a second pixel that covers a portion of the foreground area and aportion of the background area, and a third pixel that covers a portionof the background area; using input signals associated with the firstpixel to determine a distance to a first object located in theforeground area; and using input signals associated with the secondpixel and input signals associated with the third pixel to determine adistance to a second object located in the background area.

In one embodiment, a LIDAR system may comprise at least one processor.The at least one processor may be configured to control at least oneLIDAR light source in a manner enabling light projected from the atleast one light source to vary over a plurality of scans of a field ofview. The field of view may include a foreground area and a backgroundarea. The at least one processor may be further configured to receivefrom a group of detectors a plurality of input signals indicative ofreflections of the projected light from the field of view. Arepresentation of a portion of the field of view associated with aplurality of pixels is constructible from the plurality of inputsignals. The at least one processor may be further configured to detecta possible existence of an object in the background area based on firstinput signals associated with a first scanning cycle. Anobject-existence-certainty level in the first scanning cycle may bebelow a threshold. The at least one processor may be further configuredto detect a possible existence of the object based on second inputsignals associated with a second scanning cycle. Anobject-existence-certainty level in the second scanning cycle may bebelow the threshold. The at least one processor may be furtherconfigured to aggregate the first input signals associated with thefirst scanning cycle and the second input signals associated with thesecond scanning cycle to detect an existence of the object at anobject-existence-certainty level higher than the threshold.

In one embodiment, a vehicle may comprise a body and at least oneprocessor. The at least one processor may be configured to control atleast one LIDAR light source in a manner enabling light projected fromof the at least one LIDAR light source to vary over a plurality of scansof a field of view. The field of view may include a foreground area anda background area. The at least one processor may be further configuredto receive from a group of detectors a plurality of input signalsindicative of reflections of the projected light from the field of view.A representation of a portion of the field of view associated with aplurality of pixels may be constructible from the plurality of inputsignals. The at least one processor may be further configured to detecta possible existence of an object in the background area based on firstinput signals associated with a first scanning cycle. Anobject-existence-certainty level in the first scanning cycle may bebelow a threshold. The at least one processor may be further configuredto detect a possible existence of the object based on second inputsignals associated with a second scanning cycle. Anobject-existence-certainty level in the second scanning cycle may bebelow the threshold. The at least one processor may be furtherconfigured to use the first input signals associated with the firstscanning cycle and the second input signals associated with the secondscanning cycle to detect an existence of the object at anobject-existence-certainty level higher than the threshold.

In one embodiment, a method for using LIDAR system to detecting objectsin a field of view may comprise controlling at least one LIDAR lightsource in a manner enabling light flux of the at least one LIDAR lightsource to vary over a plurality of scans of a field of view. The fieldof view may include a foreground area and a background area. The methodmay further comprise receiving from a group of detectors a plurality ofinput signals indicative of reflections of light projected from thefield of view. A representation of a portion of the field of viewassociated with a plurality of pixels may be constructible from theplurality of input signals. The method may further comprise detecting apossible existence of an object in the background area based on firstinput signals associated with a first scanning cycle. Anobject-existence-certainty level in the first scanning cycle may bebelow a threshold. The method may further comprise detecting a possibleexistence of the object based on second input signals associated with asecond scanning cycle. An object-existence-certainty level in the secondscanning cycle may be below the threshold. The method may furthercomprise aggregating the first input signals associated with the firstscanning cycle and the second input signals associated with the secondscanning cycle to detect an existence of the object at anobject-existence-certainty level higher than the threshold.

One aspect of the present disclosure is directed to a LIDAR system, theLIDAR system comprises at least one processor configured to control atleast one LIDAR light source for illuminating a field of view, receivefrom a group of light detectors a plurality of input signals indicativeof reflections of light from objects in the field of view, process afirst subset of the input signals associated with a first region of thefield of view to detect a first object in the first region, whereinprocessing the first subset is performed individually on each inputsignal of the first subset of the input signals, process a second subsetof the input signals associated with a second region of the field ofview to detect at least one second object in the second region, whereinthe at least one second object is located at a greater distance from theat least one light source than the first object and wherein processingof the second subset includes processing together input signals of thesecond subset, and output information indicative of a distance to thefirst object and information indicative of a distance to the at leastone second object.

Another aspect of the present disclosure is directed to vehicle. Thevehicle comprises a body and at least one processor within the body andthe processor is configured to control at least one light source forilluminating a field of view, receive from a group of detectors aplurality of input signals indicative of reflections of light fromobjects in the field of view, process a first subset of the inputsignals associated with a first region of the field of view to detect afirst object in the first region, wherein processing the first subset isperformed individually on each of the first subset of the input signals,process a second subset of the input signals associated with a secondregion of the field of view to detect at least one second object in thesecond region, wherein the at least one second object is located at agreater distance from the at least one light source than the firstobject and wherein processing of the second subset includes combininginput signals of the second subset, and output information associatedwith a distance to the first object and information associated with adistance to the at least one second object.

Yet another aspect of the present disclosure is directed to a method forusing a LIDAR system to determine distances to objects in a field ofview, the method comprises controlling at least one light source forilluminating the field of view, receiving from a group of detectors aplurality of input signals indicative of reflections of light fromobjects in the field of view, processing a first subset of the inputsignals associated with a first region of the field of view to detect afirst object in the first region, wherein processing the first subset isperformed individually on each of the first subset of the input signals,processing a second subset of the input signals associated with a secondregion of the field of view to detect at least one second object in thesecond region, wherein the at least one second object is located at agreater distance from the light source than the first object and whereinprocessing of the second subset includes combining input signals of thesecond subset, and outputting information associated with a distance tothe first object and information associated with a distance to the atleast one second object.

One aspect of the present disclosure is directed to a LIDAR system foruse in a vehicle, the LIDAR system comprises a first housing containingat least one processor configured to control at least one light sourcein a manner enabling light flux of the at least one light source to varyover a single scan of a field of view, at least one second housingconfigured for location in the vehicle remote from the first housing,the at least one second housing containing a controllable lightdeflector configured to deflect light from the at least one lightsource, and at least one actuator configured to move the light deflectorsuch that during a single scanning cycle the light deflector movesthrough a plurality of different instantaneous positions in order toscan the field of view, and at least one data conduit configured tointerconnect the first housing and the at least one second housing, thedata conduit is associated with a forward path from the first housing tothe at least one second housing and a return path from the at least onesecond housing to the first housing, wherein the data conduit isconfigured to cooperate with the at least one processor and the at leastone actuator such that the forward path is enabled to convey controlsignals for controlling the at least one actuator and the return path isenabled to convey to the at least one processor reflections signalsindicative of light reflected from objects in the field of view.

Another aspect of the present disclosure is directed to vehicle. Thevehicle comprises a body, a first housing containing at least oneprocessor configured to control at least one light source in a mannerenabling light flux of the at least one light source to vary over scansof a field of view, a plurality of second housings configured forlocation in the vehicle remote from the first housing, the each secondhousing containing at least one light deflector configured to deflectlight from the at least one light source, and at least one actuatorconfigured to move the at least one light deflector such that during asingle scanning cycle the light deflector moves through a plurality ofdifferent instantaneous positions in order to scan the field of view,and a plurality of data conduits configured to interconnect the firsthousing with the plurality of second housings, each data conduit beingconfigured to cooperate with the at least one processor and the at leastone actuator to enable a forward path from the first housing to eachsecond housing and a return path from each second housing to the firsthousing, such that the forward path is enabled to convey control signalsfor controlling the at least one actuator and the return path is enabledto convey to the at least one processor reflections signals indicativeof light reflected from objects in the field of view.

In one aspect, a MEMS scanning device may include: a movable MEMS mirrorconfigured to pivot about at least one axis; a plurality of actuatorsconfigured to cause pivoting of the movable MEMS mirror about the atleast one axis in at least one first direction; a plurality ofrestraining springs configured to facilitate pivoting of the movableMEMS mirror about the at least one axis in at least one second directiondifferent from the at least one first direction. At least two of themovable MEMS mirror, the number of actuators, and the number ofrestraining springs may be constructed of at least two differing waferswith mechanical properties that differ from each other. The at least twodiffering wafers may be directly bonded together to form a unifiedstructure.

In another aspect, a MEMS scanning device may include: a movable MEMSmirror configured to pivot about at least one axis, the movable MEMSmirror being constructed of at least one first wafer; a plurality ofactuators configured to cause pivoting of the movable MEMS mirror aboutthe at least one axis in at least one first direction, the plurality ofactuators being constructed of at least one second wafer different fromthe at least one first wafer; a plurality of restraining springsconfigured to facilitate pivoting of the movable MEMS mirror about theat least one axis in at least one second direction different from the atleast one first direction. The plurality of restraining springs may beconstructed of at least one third wafer, different from the at least onefirst wafer and the at least one second wafer. The at least one firstwafer, the at least one second wafer, and the at least one third wafermay each have differing mechanical properties and may be bonded togetherto form a composite structure.

In another aspect, a LIDAR system may include: at least one housingmountable on a vehicle; a light source within the at least one housingconfigured to project light for illuminating an object in an environmentof the vehicle; a scanning unit configured to deflect light from thelight source in order to scan at least part of the environment of thevehicle; at least one sensor within the at least one housing configuredto detect reflections of the projected light; and at least one processorconfigured to determine a distance between the vehicle and the object.The scanning unit may include: a movable MEMS mirror configured to pivotabout at least one axis; a plurality of actuators configured to causepivoting of the movable MEMS mirror about the at least one axis in atleast one first direction; a plurality of restraining springs configuredto facilitate pivoting of the movable MEMS mirror about the at least oneaxis in a second direction different from the first direction. At leasttwo of: the movable MEMS mirror, the plurality of actuators, and theplurality of restraining springs, may be constructed of at least twodiffering wafers with mechanical properties that differ from each other.At least two differing wafers may be directly bonded together to form aunified structure.

A MEMS scanning device may include a movable MEMS mirror configured topivot about at least one axis; at least one actuator operable to rotatethe MEMS mirror about the at least one axis, each actuator out of the atleast one actuator operable to bend upon actuation to move the MEMSmirror; and at least one flexible interconnect element coupled betweenthe at least one actuator and the MEMS mirror for transferring thepulling force of the bending of the at least one actuator to the MEMSmirror. Each flexible interconnect element out of the at least oneinterconnect element may include an elongated structure comprising atleast two turns at opposing directions, each turn greater than 120°.

A polygon scanner assembly may include an at least-partly transparenttank; an at least-partly transparent fluid, confined within the tank;and a reflective polygon, at least partly immersed in the fluid, thereflective polygon operable to reflect an incidence beam of lightarriving from outside the tank to provide a deflected beam of lightexiting from the tank outward; wherein a shape of an exterior wall ofthe tank is not parallel to a shape of an interior wall of the wall inat least a transference part of the wall of the tank through which atleast one of the incidence beam and the deflected beam propagates.

Other embodiments may include non-transitory computer-readable mediastoring instructions that, when executed by at least one processor,cause the at least one processor to execute one or more methodsdisclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this disclosure, illustrate various disclosed embodiments. Inthe drawings:

FIG. 1A is a diagram illustrating an exemplary LIDAR system consistentwith disclosed embodiments.

FIG. 1B is an image showing an exemplary output of single scanning cycleof a LIDAR system mounted on a vehicle consistent with disclosedembodiments.

FIG. 1C is another image showing a representation of a point cloud modeldetermined from output of a LIDAR system consistent with disclosedembodiments.

FIGS. 2A-2G are diagrams illustrating different configurations ofprojecting units in accordance with some embodiments of the presentdisclosure.

FIGS. 3A-3D are diagrams illustrating different configurations ofscanning units in accordance with some embodiments of the presentdisclosure.

FIGS. 4A-4E are diagrams illustrating different configurations ofsensing units in accordance with some embodiments of the presentdisclosure.

FIG. 5A includes four example diagrams illustrating emission patterns ina single frame-time for a single portion of the field of view.

FIG. 5B includes three example diagrams illustrating emission scheme ina single frame-time for the whole field of view.

FIG. 5C is a diagram illustrating the actual light emission projectedtowards and reflections received during a single frame-time for thewhole field of view.

FIGS. 6A-6C are diagrams illustrating a first example implementationconsistent with some embodiments of the present disclosure.

FIG. 6D is a diagram illustrating a second example implementationconsistent with some embodiments of the present disclosure.

FIGS. 7A and 7B include graphs of exemplary parasitic pulses that may beencountered by a LIDAR system.

FIG. 8A includes a diagrammatic representation of a LIDAR scanning anddetection system, according to exemplary disclosed embodiments.

FIG. 8B includes a diagrammatic representation of a LIDAR scanning anddetection system, according to exemplary disclosed embodiments.

FIG. 8C includes a diagrammatic representation of a light spot incidentupon a detector, according to exemplary disclosed embodiments.

FIG. 8D includes a diagrammatic representation of light intensity of thespot shown in FIG. 8C taken across the line A-A of FIG. 8C.

FIG. 8E diagrammatically illustrates light components incident uponsensors 806 and 808.

FIG. 9 includes a flowchart representation of a method, according toexemplary disclosed embodiments.

FIG. 10A is a diagram illustrating an exemplary obstruction detectionsystem consistent with disclosed embodiments.

FIG. 10B is a diagram illustrating an exemplary obstruction detectionsystem consistent with disclosed embodiments.

FIG. 10C is a block diagram of an exemplary processor included in anobstruction detection system consistent with disclosed embodiments.

FIG. 10D is a diagram illustrating an exemplary process for obstructionclassification consistent with disclosed embodiments.

FIG. 10E is a diagram illustrating an exemplary process for obstructionclassification consistent with disclosed embodiments.

FIG. 10F is illustrates exemplary obstruction patterns consistent withdisclosed embodiments.

FIG. 10G is an exemplary graph of three types of returning signalsconsistent with disclosed embodiments.

FIG. 11 is a flow chart of an exemplary method for detecting andclassifying obstructions.

FIG. 12 is a flow chart of an exemplary process for detecting andclassifying obstructions consistent with disclosed embodiments.

FIG. 13A is a diagram illustrating an exemplary LIDAR system having forcapturing information relative to a plurality of field of view pixelsconsistent with disclosed embodiments.

FIG. 13B is a diagram illustrating an exemplary LIDAR system having forcapturing information relative to a plurality of field of view pixelsconsistent with disclosed embodiments.

FIG. 14A is a diagram representative of a background field of viewpixel, an intermediate field of view pixel, and a foreground field ofview pixel imaged by an exemplary LIDAR system consistent with disclosedembodiments.

FIG. 14B is a diagram illustrating a front-facing view of the pixels ofFIG. 14A.

FIG. 14C is a diagram illustrating an exemplary vehicle with a LIDARsystem for capturing information relative to at least three FOV pixelsconsistent with disclosed embodiments.

FIG. 14D is a diagram illustrating an exemplary vehicle with a LIDARsystem for capturing information relative to at least three FOV pixelsto detect a foreground object and a background object consistent withdisclosed embodiments.

FIG. 14E is a diagram illustrating an exemplary vehicle with a LIDARsystem for capturing information relative to at least three FOV pixelsto detect a foreground object partially occluding a background objectconsistent with disclosed embodiments.

FIG. 14F is a diagram illustrating an exemplary vehicle with a LIDARsystem for capturing information relative to at least four FOV pixels todetect a foreground object and a background object consistent withdisclosed embodiments.

FIG. 15 is a diagram illustrating a flowchart of an exemplary method foraggregating pixel data from a plurality of pixels consistent withdisclosed embodiments.

FIG. 16 is a diagram illustrating an exemplary vehicle with a LIDARsystem using at least two scan cycles to detect objects consistent withdisclosed embodiments.

FIG. 17A is a diagram illustrating adjusting an operating parameterbetween scan cycles consistent with disclosed embodiments.

FIG. 17B is a diagram illustrating using a plurality of thresholds totrigger signal aggregation consistent with disclosed embodiments.

FIG. 17C is a diagram illustrating using a threshold to trigger furthersignal aggregation consistent with disclosed embodiments.

FIG. 17D is a diagram illustrating using further signal aggregation toimprove resolution consistent with disclosed embodiments.

FIG. 17E is a diagram illustrating an exemplary vehicle with a LIDARsystem using measured properties across scan cycles to detect objectsconsistent with disclosed embodiments.

FIG. 17F is a diagram illustrating an exemplary vehicle with a LIDARsystem using detected motion across scan cycles to detect objectsconsistent with disclosed embodiments.

FIG. 18A is a diagram illustrating a flowchart of an exemplary methodfor aggragating pixel data over scan cycles to improve object detectionconsistent with disclosed embodiments.

FIG. 18B is a diagram illustrating a flowchart of an exemplary methodfor aggragating pixel data over scan cycles to improve objectclassification consistent with disclosed embodiments.

FIG. 19A is a diagram illustrating an exemplary scene including objectsin an environment of a LIDAR system.

FIG. 19B is a diagram illustrating an exemplary scene including a nearfield object and a far field object in a field of view of the LIDARsystem.

FIG. 19C is a diagram illustrating an exemplary sensor response toreflected light received during a scan of a portion of the LIDAR FOVdepicted in FIG. 19B.

FIG. 19D is a diagram illustrating an exemplary LIDAR FOV, according todisclosed embodiments.

FIG. 20A is a diagram illustrating sensor pixel outputs and an examplebinning technique, according to disclosed embodiments.

FIG. 20B is a diagram illustrating sensor pixel outputs and anotherexample binning technique, according to disclosed embodiments.

FIG. 21 provides a flowchart representation of a binning methodaccording to exemplary disclosed embodiments.

FIG. 22 is a diagram showing an exemplary LIDAR system with distributedLIDAR system components in accordance with some embodiments of thepresent disclosure.

FIG. 23 is a diagram showing an exemplary LIDAR system with distributedLIDAR system components in which a first housing includes at least onesensor in accordance with some embodiments of the present disclosure.

FIG. 24 is a diagram showing an exemplary LIDAR system with distributedLIDAR system components in which at least one second housing includes atleast one sensor in accordance with some embodiments of the presentdisclosure.

FIG. 25 is an illustration of a MEMS scanning device, consistent withdisclosed embodiments.

FIGS. 26A-26C are illustrations of a cross-sectional area of a MEMSscanning device, consistent with disclosed embodiments.

FIG. 27A-27C are illustrations of exemplary configurations of a MEMSscanning device, consistent with disclosed embodiments.

FIG. 28 is an illustration of an exemplary configuration of a MEMSscanning device, consistent with disclosed embodiments.

FIG. 29 provides a diagrammatic representation of a MEMS scanning devicein accordance with examples of the presently disclosed subject matter.

FIGS. 30A-30F provide diagrammatic representations of examples ofreflective scanning polygons in accordance with examples of thepresently disclosed subject matter.

FIGS. 31A-31C provide diagrammatic representations of examples ofpolygon scanners in accordance with examples of the presently disclosedsubject matter.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings.Wherever possible, the same reference numbers are used in the drawingsand the following description to refer to the same or similar parts.While several illustrative embodiments are described herein,modifications, adaptations and other implementations are possible. Forexample, substitutions, additions or modifications may be made to thecomponents illustrated in the drawings, and the illustrative methodsdescribed herein may be modified by substituting, reordering, removing,or adding steps to the disclosed methods. Accordingly, the followingdetailed description is not limited to the disclosed embodiments andexamples. Instead, the proper scope is defined by the appended claims.

Terms Definitions

Disclosed embodiments may involve an optical system. As used herein, theterm “optical system” broadly includes any system that is used for thegeneration, detection and/or manipulation of light. By way of exampleonly, an optical system may include one or more optical components forgenerating, detecting and/or manipulating light. For example, lightsources, lenses, mirrors, prisms, beam splitters, collimators,polarizing optics, optical modulators, optical switches, opticalamplifiers, optical detectors, optical sensors, fiber optics,semiconductor optic components, while each not necessarily required, mayeach be part of an optical system. In addition to the one or moreoptical components, an optical system may also include other non-opticalcomponents such as electrical components, mechanical components,chemical reaction components, and semiconductor components. Thenon-optical components may cooperate with optical components of theoptical system. For example, the optical system may include at least oneprocessor for analyzing detected light.

Consistent with the present disclosure, the optical system may be aLIDAR system. As used herein, the term “LIDAR system” broadly includesany system which can determine values of parameters indicative of adistance between a pair of tangible objects based on reflected light. Inone embodiment, the LIDAR system may determine a distance between a pairof tangible objects based on reflections of light emitted by the LIDARsystem. As used herein, the term “determine distances” broadly includesgenerating outputs which are indicative of distances between pairs oftangible objects. The determined distance may represent the physicaldimension between a pair of tangible objects. By way of example only,the determined distance may include a line of flight distance betweenthe LIDAR system and another tangible object in a field of view of theLIDAR system. In another embodiment, the LIDAR system may determine therelative velocity between a pair of tangible objects based onreflections of light emitted by the LIDAR system. Examples of outputsindicative of the distance between a pair of tangible objects include: anumber of standard length units between the tangible objects (e.g.number of meters, number of inches, number of kilometers, number ofmillimeters), a number of arbitrary length units (e.g. number of LIDARsystem lengths), a ratio between the distance to another length (e.g. aratio to a length of an object detected in a field of view of the LIDARsystem), an amount of time (e.g. given as standard unit, arbitrary unitsor ratio, for example, the time it takes light to travel between thetangible objects), one or more locations (e.g. specified using an agreedcoordinate system, specified in relation to a known location), and more.

The LIDAR system may determine the distance between a pair of tangibleobjects based on reflected light. In one embodiment, the LIDAR systemmay process detection results of a sensor which creates temporalinformation indicative of a period of time between the emission of alight signal and the time of its detection by the sensor. The period oftime is occasionally referred to as “time of flight” of the lightsignal. In one example, the light signal may be a short pulse, whoserise and/or fall time may be detected in reception. Using knowninformation about the speed of light in the relevant medium (usuallyair), the information regarding the time of flight of the light signalcan be processed to provide the distance the light signal traveledbetween emission and detection. In another embodiment, the LIDAR systemmay determine the distance based on frequency phase-shift (or multiplefrequency phase-shift). Specifically, the LIDAR system may processinformation indicative of one or more modulation phase shifts (e.g. bysolving some simultaneous equations to give a final measure) of thelight signal. For example, the emitted optical signal may be modulatedwith one or more constant frequencies. The at least one phase shift ofthe modulation between the emitted signal and the detected reflectionmay be indicative of the distance the light traveled between emissionand detection. The modulation may be applied to a continuous wave lightsignal, to a quasi-continuous wave light signal, or to another type ofemitted light signal. It is noted that additional information may beused by the LIDAR system for determining the distance, e.g. locationinformation (e.g. relative positions) between the projection location,the detection location of the signal (especially if distanced from oneanother), and more.

In some embodiments, the LIDAR system may be used for detecting aplurality of objects in an environment of the LIDAR system. The term“detecting an object in an environment of the LIDAR system” broadlyincludes generating information which is indicative of an object thatreflected light toward a detector associated with the LIDAR system. Ifmore than one object is detected by the LIDAR system, the generatedinformation pertaining to different objects may be interconnected, forexample a car is driving on a road, a bird is sitting on the tree, a mantouches a bicycle, a van moves towards a building. The dimensions of theenvironment in which the LIDAR system detects objects may vary withrespect to implementation. For example, the LIDAR system may be used fordetecting a plurality of objects in an environment of a vehicle on whichthe LIDAR system is installed, up to a horizontal distance of 100 m (or200 m, 300 m, etc.), and up to a vertical distance of 10 m (or 25 m, 50m, etc.). In another example, the LIDAR system may be used for detectinga plurality of objects in an environment of a vehicle or within apredefined horizontal range (e.g., 25°, 50°, 100°, 180°, etc.), and upto a predefined vertical elevation (e.g., ±10°, ±20°, +40°-20°, ±90° or0°-90°).

As used herein, the term “detecting an object” may broadly refer todetermining an existence of the object (e.g., an object may exist in acertain direction with respect to the LIDAR system and/or to anotherreference location, or an object may exist in a certain spatial volume).Additionally or alternatively, the term “detecting an object” may referto determining a distance between the object and another location (e.g.a location of the LIDAR system, a location on earth, or a location ofanother object). Additionally or alternatively, the term “detecting anobject” may refer to identifying the object (e.g. classifying a type ofobject such as car, plant, tree, road; recognizing a specific object(e.g., the Washington Monument); determining a license plate number;determining a composition of an object (e.g., solid, liquid,transparent, semitransparent); determining a kinematic parameter of anobject (e.g., whether it is moving, its velocity, its movementdirection, expansion of the object). Additionally or alternatively, theterm “detecting an object” may refer to generating a point cloud map inwhich every point of one or more points of the point cloud mapcorrespond to a location in the object or a location on a face thereof.In one embodiment, the data resolution associated with the point cloudmap representation of the field of view may be associated with 0.1°×0.1°or 0.3°×0.3° of the field of view.

Consistent with the present disclosure, the term “object” broadlyincludes a finite composition of matter that may reflect light from atleast a portion thereof. For example, an object may be at leastpartially solid (e.g. cars, trees); at least partially liquid (e.g.puddles on the road, rain); at least partly gaseous (e.g. fumes,clouds); made from a multitude of distinct particles (e.g. sand storm,fog, spray); and may be of one or more scales of magnitude, such as ˜1millimeter (mm), ˜5 mm, ˜10 mm, ˜50 mm, ˜100 mm, ˜500 mm, ˜1 meter (m),˜5 m, ˜10 m, ˜50 m, ˜100 m, and so on. Smaller or larger objects, aswell as any size in between those examples, may also be detected. It isnoted that for various reasons, the LIDAR system may detect only part ofthe object. For example, in some cases, light may be reflected from onlysome sides of the object (e.g., only the side opposing the LIDAR systemwill be detected); in other cases, light may be projected on only partof the object (e.g. laser beam projected onto a road or a building); inother cases, the object may be partly blocked by another object betweenthe LIDAR system and the detected object; in other cases, the LIDAR'ssensor may only detects light reflected from a portion of the object,e.g., because ambient light or other interferences interfere withdetection of some portions of the object.

Consistent with the present disclosure, a LIDAR system may be configuredto detect objects by scanning the environment of LIDAR system. The term“scanning the environment of LIDAR system” broadly includes illuminatingthe field of view or a portion of the field of view of the LIDAR system.In one example, scanning the environment of LIDAR system may be achievedby moving or pivoting a light deflector to deflect light in differingdirections toward different parts of the field of view. In anotherexample, scanning the environment of LIDAR system may be achieved bychanging a positioning (i.e. location and/or orientation) of a sensorwith respect to the field of view. In another example, scanning theenvironment of LIDAR system may be achieved by changing a positioning(i.e. location and/or orientation) of a light source with respect to thefield of view. In yet another example, scanning the environment of LIDARsystem may be achieved by changing the positions of at least one lightsource and of at least one sensor to move rigidly respect to the fieldof view (i.e. the relative distance and orientation of the at least onesensor and of the at least one light source remains).

As used herein the term “field of view of the LIDAR system” may broadlyinclude an extent of the observable environment of LIDAR system in whichobjects may be detected. It is noted that the field of view (FOV) of theLIDAR system may be affected by various conditions such as but notlimited to: an orientation of the LIDAR system (e.g. is the direction ofan optical axis of the LIDAR system); a position of the LIDAR systemwith respect to the environment (e.g. distance above ground and adjacenttopography and obstacles); operational parameters of the LIDAR system(e.g. emission power, computational settings, defined angles ofoperation), etc. The field of view of LIDAR system may be defined, forexample, by a solid angle (e.g. defined using 4, 0 angles, in which 4and 0 are angles defined in perpendicular planes, e.g. with respect tosymmetry axes of the LIDAR system and/or its FOV). In one example, thefield of view may also be defined within a certain range (e.g. up to 200m).

Similarly, the term “instantaneous field of view” may broadly include anextent of the observable environment in which objects may be detected bythe LIDAR system at any given moment. For example, for a scanning LIDARsystem, the instantaneous field of view is narrower than the entire FOVof the LIDAR system, and it can be moved within the FOV of the LIDARsystem in order to enable detection in other parts of the FOV of theLIDAR system. The movement of the instantaneous field of view within theFOV of the LIDAR system may be achieved by moving a light deflector ofthe LIDAR system (or external to the LIDAR system), so as to deflectbeams of light to and/or from the LIDAR system in differing directions.In one embodiment, LIDAR system may be configured to scan scene in theenvironment in which the LIDAR system is operating. As used herein theterm “scene” may broadly include some or all of the objects within thefield of view of the LIDAR system, in their relative positions and intheir current states, within an operational duration of the LIDARsystem. For example, the scene may include ground elements (e.g. earth,roads, grass, sidewalks, road surface marking), sky, man-made objects(e.g. vehicles, buildings, signs), vegetation, people, animals, lightprojecting elements (e.g. flashlights, sun, other LIDAR systems), and soon.

Disclosed embodiments may involve obtaining information for use ingenerating reconstructed three-dimensional models. Examples of types ofreconstructed three-dimensional models which may be used include pointcloud models, and Polygon Mesh (e.g. a triangle mesh). The terms “pointcloud” and “point cloud model” are widely known in the art, and shouldbe construed to include a set of data points located spatially in somecoordinate system (i.e., having an identifiable location in a spacedescribed by a respective coordinate system). The term “point cloudpoint” refer to a point in space (which may be dimensionless, or aminiature cellular space, e.g. 1 cm3), and whose location may bedescribed by the point cloud model using a set of coordinates (e.g.(X,Y,Z), (r,ϕ,θ)). By way of example only, the point cloud model maystore additional information for some or all of its points (e.g. colorinformation for points generated from camera images). Likewise, anyother type of reconstructed three-dimensional model may store additionalinformation for some or all of its objects. Similarly, the terms“polygon mesh” and “triangle mesh” are widely known in the art, and areto be construed to include, among other things, a set of vertices, edgesand faces that define the shape of one or more 3D objects (such as apolyhedral object). The faces may include one or more of the following:triangles (triangle mesh), quadrilaterals, or other simple convexpolygons, since this may simplify rendering. The faces may also includemore general concave polygons, or polygons with holes. Polygon meshesmay be represented using differing techniques, such as: Vertex-vertexmeshes, Face-vertex meshes, Winged-edge meshes and Render dynamicmeshes. Different portions of the polygon mesh (e.g., vertex, face,edge) are located spatially in some coordinate system (i.e., having anidentifiable location in a space described by the respective coordinatesystem), either directly and/or relative to one another. The generationof the reconstructed three-dimensional model may be implemented usingany standard, dedicated and/or novel photogrammetry technique, many ofwhich are known in the art. It is noted that other types of models ofthe environment may be generated by the LIDAR system.

Consistent with disclosed embodiments, the LIDAR system may include atleast one projecting unit with a light source configured to projectlight. As used herein the term “light source” broadly refers to anydevice configured to emit light. In one embodiment, the light source maybe a laser such as a solid-state laser, laser diode, a high power laser,or an alternative light source such as, a light emitting diode(LED)-based light source. In addition, light source 112 as illustratedthroughout the figures, may emit light in differing formats, such aslight pulses, continuous wave (CW), quasi-CW, and so on. For example,one type of light source that may be used is a vertical-cavitysurface-emitting laser (VCSEL). Another type of light source that may beused is an external cavity diode laser (ECDL). In some examples, thelight source may include a laser diode configured to emit light at awavelength between about 650 nm and 1150 nm. Alternatively, the lightsource may include a laser diode configured to emit light at awavelength between about 800 nm and about 1000 nm, between about 850 nmand about 950 nm, or between about 1300 nm and about 1600 nm. Unlessindicated otherwise, the term “about” with regards to a numeric value isdefined as a variance of up to 5% with respect to the stated value.Additional details on the projecting unit and the at least one lightsource are described below with reference to FIGS. 2A-2C.

Consistent with disclosed embodiments, the LIDAR system may include atleast one scanning unit with at least one light deflector configured todeflect light from the light source in order to scan the field of view.The term “light deflector” broadly includes any mechanism or modulewhich is configured to make light deviate from its original path; forexample, a mirror, a prism, controllable lens, a mechanical mirror,mechanical scanning polygons, active diffraction (e.g. controllableLCD), Risley prisms, non-mechanical-electro-optical beam steering (suchas made by Vscent), polarization grating (such as offered by BoulderNon-Linear Systems), optical phased array (OPA), and more. In oneembodiment, a light deflector may include a plurality of opticalcomponents, such as at least one reflecting element (e.g. a mirror), atleast one refracting element (e.g. a prism, a lens), and so on. In oneexample, the light deflector may be movable, to cause light deviate todiffering degrees (e.g. discrete degrees, or over a continuous span ofdegrees). The light deflector may optionally be controllable indifferent ways (e.g. deflect to a degree α, change deflection angle byΔα, move a component of the light deflector by M millimeters, changespeed in which the deflection angle changes). In addition, the lightdeflector may optionally be operable to change an angle of deflectionwithin a single plane (e.g., θ coordinate). The light deflector mayoptionally be operable to change an angle of deflection within twonon-parallel planes (e.g., θ and ϕ coordinates). Alternatively or inaddition, the light deflector may optionally be operable to change anangle of deflection between predetermined settings (e.g. along apredefined scanning route) or otherwise. With respect the use of lightdeflectors in LIDAR systems, it is noted that a light deflector may beused in the outbound direction (also referred to as transmissiondirection, or TX) to deflect light from the light source to at least apart of the field of view. However, a light deflector may also be usedin the inbound direction (also referred to as reception direction, orRX) to deflect light from at least a part of the field of view to one ormore light sensors. Additional details on the scanning unit and the atleast one light deflector are described below with reference to FIGS.3A-3C.

Disclosed embodiments may involve pivoting the light deflector in orderto scan the field of view. As used herein the term “pivoting” broadlyincludes rotating of an object (especially a solid object) about one ormore axis of rotation, while substantially maintaining a center ofrotation fixed. In one embodiment, the pivoting of the light deflectormay include rotation of the light deflector about a fixed axis (e.g., ashaft), but this is not necessarily so. For example, in some MEMS mirrorimplementation, the MEMS mirror may move by actuation of a plurality ofbenders connected to the mirror, the mirror may experience some spatialtranslation in addition to rotation. Nevertheless, such mirror may bedesigned to rotate about a substantially fixed axis, and thereforeconsistent with the present disclosure it considered to be pivoted. Inother embodiments, some types of light deflectors (e.g.non-mechanical-electro-optical beam steering, OPA) do not require anymoving components or internal movements in order to change thedeflection angles of deflected light. It is noted that any discussionrelating to moving or pivoting a light deflector is also mutatismutandis applicable to controlling the light deflector such that itchanges a deflection behavior of the light deflector. For example,controlling the light deflector may cause a change in a deflection angleof beams of light arriving from at least one direction.

Disclosed embodiments may involve receiving reflections associated witha portion of the field of view corresponding to a single instantaneousposition of the light deflector. As used herein, the term “instantaneousposition of the light deflector” (also referred to as “state of thelight deflector”) broadly refers to the location or position in spacewhere at least one controlled component of the light deflector issituated at an instantaneous point in time, or over a short span oftime. In one embodiment, the instantaneous position of light deflectormay be gauged with respect to a frame of reference. The frame ofreference may pertain to at least one fixed point in the LIDAR system.Or, for example, the frame of reference may pertain to at least onefixed point in the scene. In some embodiments, the instantaneousposition of the light deflector may include some movement of one or morecomponents of the light deflector (e.g. mirror, prism), usually to alimited degree with respect to the maximal degree of change during ascanning of the field of view. For example, a scanning of the entire thefield of view of the LIDAR system may include changing deflection oflight over a span of 30°, and the instantaneous position of the at leastone light deflector may include angular shifts of the light deflectorwithin 0.05°. In other embodiments, the term “instantaneous position ofthe light deflector” may refer to the positions of the light deflectorduring acquisition of light which is processed to provide data for asingle point of a point cloud (or another type of 3D model) generated bythe LIDAR system. In some embodiments, an instantaneous position of thelight deflector may correspond with a fixed position or orientation inwhich the deflector pauses for a short time during illumination of aparticular sub-region of the LIDAR field of view. In other cases, aninstantaneous position of the light deflector may correspond with acertain position/orientation along a scanned range ofpositions/orientations of the light deflector that the light deflectorpasses through as part of a continuous or semi-continuous scan of theLIDAR field of view. In some embodiments, the light deflector may bemoved such that during a scanning cycle of the LIDAR FOV the lightdeflector is located at a plurality of different instantaneouspositions. In other words, during the period of time in which a scanningcycle occurs, the deflector may be moved through a series of differentinstantaneous positions/orientations, and the deflector may reach eachdifferent instantaneous position/orientation at a different time duringthe scanning cycle.

Consistent with disclosed embodiments, the LIDAR system may include atleast one sensing unit with at least one sensor configured to detectreflections from objects in the field of view. The term “sensor” broadlyincludes any device, element, or system capable of measuring properties(e.g., power, frequency, phase, pulse timing, pulse duration) ofelectromagnetic waves and to generate an output relating to the measuredproperties. In some embodiments, the at least one sensor may include aplurality of detectors constructed from a plurality of detectingelements. The at least one sensor may include light sensors of one ormore types. It is noted that the at least one sensor may includemultiple sensors of the same type which may differ in othercharacteristics (e.g., sensitivity, size). Other types of sensors mayalso be used. Combinations of several types of sensors can be used fordifferent reasons, such as improving detection over a span of ranges(especially in close range); improving the dynamic range of the sensor;improving the temporal response of the sensor; and improving detectionin varying environmental conditions (e.g. atmospheric temperature, rain,etc.).

In one embodiment, the at least one sensor includes a SiPM (Siliconphotomultipliers) which is a solid-state single-photon-sensitive devicebuilt from an array of avalanche photodiode (APD), single photonavalanche diode (SPAD), serving as detection elements on a commonsilicon substrate. In one example, a typical distance between SPADs maybe between about 10 μm and about 50 μm, wherein each SPAD may have arecovery time of between about 20 ns and about 100 ns. Similarphotomultipliers from other, non-silicon materials may also be used.Although a SiPM device works in digital/switching mode, the SiPM is ananalog device because all the microcells may be read in parallel, makingit possible to generate signals within a dynamic range from a singlephoton to hundreds and thousands of photons detected by the differentSPADs. It is noted that outputs from different types of sensors (e.g.,SPAD, APD, SiPM, PIN diode, Photodetector) may be combined together to asingle output which may be processed by a processor of the LIDAR system.Additional details on the sensing unit and the at least one sensor aredescribed below with reference to FIGS. 4A-4C.

Consistent with disclosed embodiments, the LIDAR system may include orcommunicate with at least one processor configured to execute differingfunctions. The at least one processor may constitute any physical devicehaving an electric circuit that performs a logic operation on input orinputs. For example, the at least one processor may include one or moreintegrated circuits (IC), including Application-specific integratedcircuit (ASIC), microchips, microcontrollers, microprocessors, all orpart of a central processing unit (CPU), graphics processing unit (GPU),digital signal processor (DSP), field-programmable gate array (FPGA), orother circuits suitable for executing instructions or performing logicoperations. The instructions executed by at least one processor may, forexample, be pre-loaded into a memory integrated with or embedded intothe controller or may be stored in a separate memory. The memory maycomprise a Random Access Memory (RAM), a Read-Only Memory (ROM), a harddisk, an optical disk, a magnetic medium, a flash memory, otherpermanent, fixed, or volatile memory, or any other mechanism capable ofstoring instructions. In some embodiments, the memory is configured tostore information representative data about objects in the environmentof the LIDAR system. In some embodiments, the at least one processor mayinclude more than one processor. Each processor may have a similarconstruction or the processors may be of differing constructions thatare electrically connected or disconnected from each other. For example,the processors may be separate circuits or integrated in a singlecircuit. When more than one processor is used, the processors may beconfigured to operate independently or collaboratively. The processorsmay be coupled electrically, magnetically, optically, acoustically,mechanically or by other means that permit them to interact. Additionaldetails on the processing unit and the at least one processor aredescribed below with reference to FIGS. 5A-5C.

System Overview

FIG. 1A illustrates a LIDAR system 100 including a projecting unit 102,a scanning unit 104, a sensing unit 106, and a processing unit 108.LIDAR system 100 may be mountable on a vehicle 110. Consistent withembodiments of the present disclosure, projecting unit 102 may includeat least one light source 112, scanning unit 104 may include at leastone light deflector 114, sensing unit 106 may include at least onesensor 116, and processing unit 108 may include at least one processor118. In one embodiment, at least one processor 118 may be configured tocoordinate operation of the at least one light source 112 with themovement of at least one light deflector 114 in order to scan a field ofview 120. During a scanning cycle, each instantaneous position of atleast one light deflector 114 may be associated with a particularportion 122 of field of view 120. In addition, LIDAR system 100 mayinclude at least one optional optical window 124 for directing lightprojected towards field of view 120 and/or receiving light reflectedfrom objects in field of view 120. Optional optical window 124 may servedifferent purposes, such as collimation of the projected light andfocusing of the reflected light. In one embodiment, optional opticalwindow 124 may be an opening, a flat window, a lens, or any other typeof optical window.

Consistent with the present disclosure, LIDAR system 100 may be used inautonomous or semi-autonomous road-vehicles (for example, cars, buses,vans, trucks and any other terrestrial vehicle). Autonomousroad-vehicles with LIDAR system 100 may scan their environment and driveto a destination vehicle without human input. Similarly, LIDAR system100 may also be used in autonomous/semi-autonomous aerial-vehicles (forexample, UAV, drones, quadcopters, and any other airborne vehicle ordevice); or in an autonomous or semi-autonomous water vessel (e.g.,boat, ship, submarine, or any other watercraft). Autonomousaerial-vehicles and water craft with LIDAR system 100 may scan theirenvironment and navigate to a destination autonomously or using a remotehuman operator. According to one embodiment, vehicle 110 (either aroad-vehicle, aerial-vehicle, or watercraft) may use LIDAR system 100 toaid in detecting and scanning the environment in which vehicle 110 isoperating.

It should be noted that LIDAR system 100 or any of its components may beused together with any of the example embodiments and methods disclosedherein. Further, while some aspects of LIDAR system 100 are describedrelative to an exemplary vehicle-based LIDAR platform, LIDAR system 100,any of its components, or any of the processes described herein may beapplicable to LIDAR systems of other platform types.

In some embodiments, LIDAR system 100 may include one or more scanningunits 104 to scan the environment around vehicle 110. LIDAR system 100may be attached or mounted to any part of vehicle 110. Sensing unit 106may receive reflections from the surroundings of vehicle 110, andtransfer reflections signals indicative of light reflected from objectsin field of view 120 to processing unit 108. Consistent with the presentdisclosure, scanning units 104 may be mounted to or incorporated into abumper, a fender, a side panel, a spoiler, a roof, a headlight assembly,a taillight assembly, a rear-view mirror assembly, a hood, a trunk orany other suitable part of vehicle 110 capable of housing at least aportion of the LIDAR system. In some cases, LIDAR system 100 may capturea complete surround view of the environment of vehicle 110. Thus, LIDARsystem 100 may have a 360-degree horizontal field of view. In oneexample, as shown in FIG. 1A, LIDAR system 100 may include a singlescanning unit 104 mounted on a roof vehicle 110. Alternatively, LIDARsystem 100 may include multiple scanning units (e.g., two, three, four,or more scanning units 104) each with a field of few such that in theaggregate the horizontal field of view is covered by a 360-degree scanaround vehicle 110. One skilled in the art will appreciate that LIDARsystem 100 may include any number of scanning units 104 arranged in anymanner, each with an 80° to 120° field of view or less, depending on thenumber of units employed. Moreover, a 360-degree horizontal field ofview may be also obtained by mounting a multiple LIDAR systems 100 onvehicle 110, each with a single scanning unit 104. It is neverthelessnoted, that the one or more LIDAR systems 100 do not have to provide acomplete 360° field of view, and that narrower fields of view may beuseful in some situations. For example, vehicle 110 may require a firstLIDAR system 100 having an field of view of 75° looking ahead of thevehicle, and possibly a second LIDAR system 100 with a similar FOVlooking backward (optionally with a lower detection range). It is alsonoted that different vertical field of view angles may also beimplemented.

FIG. 1B is an image showing an exemplary output from a single scanningcycle of LIDAR system 100 mounted on vehicle 110 consistent withdisclosed embodiments. In this example, scanning unit 104 isincorporated into a right headlight assembly of vehicle 110. Every graydot in the image corresponds to a location in the environment aroundvehicle 110 determined from reflections detected by sensing unit 106. Inaddition to location, each gray dot may also be associated withdifferent types of information, for example, intensity (e.g., how muchlight returns back from that location), reflectivity, proximity to otherdots, and more. In one embodiment, LIDAR system 100 may generate aplurality of point-cloud data entries from detected reflections ofmultiple scanning cycles of the field of view to enable, for example,determining a point cloud model of the environment around vehicle 110.

FIG. 1C is an image showing a representation of the point cloud modeldetermined from the output of LIDAR system 100. Consistent withdisclosed embodiments, by processing the generated point-cloud dataentries of the environment around vehicle 110, a surround-view image maybe produced from the point cloud model. In one embodiment, the pointcloud model may be provided to a feature extraction module, whichprocesses the point cloud information to identify a plurality offeatures. Each feature may include data about different aspects of thepoint cloud and/or of objects in the environment around vehicle 110(e.g. cars, trees, people, and roads). Features may have the sameresolution of the point cloud model (i.e. having the same number of datapoints, optionally arranged into similar sized 2D arrays), or may havedifferent resolutions. The features may be stored in any kind of datastructure (e.g. raster, vector, 2D array, 1D array). In addition,virtual features, such as a representation of vehicle 110, border lines,or bounding boxes separating regions or objects in the image (e.g., asdepicted in FIG. 1B), and icons representing one or more identifiedobjects, may be overlaid on the representation of the point cloud modelto form the final surround-view image. For example, a symbol of vehicle110 may be overlaid at a center of the surround-view image.

The Projecting Unit

FIGS. 2A-2G depict various configurations of projecting unit 102 and itsrole in LIDAR system 100. Specifically, FIG. 2A is a diagramillustrating projecting unit 102 with a single light source; FIG. 2B isa diagram illustrating a plurality of projecting units 102 with aplurality of light sources aimed at a common light deflector 114; FIG.2C is a diagram illustrating projecting unit 102 with a primary and asecondary light sources 112; FIG. 2D is a diagram illustrating anasymmetrical deflector used in some configurations of projecting unit102; FIG. 2E is a diagram illustrating a first configuration of anon-scanning LIDAR system; FIG. 2F is a diagram illustrating a secondconfiguration of a non-scanning LIDAR system; and FIG. 2G is a diagramillustrating a LIDAR system that scans in the outbound direction anddoes not scan in the inbound direction. One skilled in the art willappreciate that the depicted configurations of projecting unit 102 mayhave numerous variations and modifications.

FIG. 2A illustrates an example of a bi-static configuration of LIDARsystem 100 in which projecting unit 102 includes a single light source112. The term “bi-static configuration” broadly refers to LIDAR systemsconfigurations in which the projected light exiting the LIDAR system andthe reflected light entering the LIDAR system pass through substantiallydifferent optical paths. In some embodiments, a bi-static configurationof LIDAR system 100 may include a separation of the optical paths byusing completely different optical components, by using parallel but notfully separated optical components, or by using the same opticalcomponents for only part of the of the optical paths (optical componentsmay include, for example, windows, lenses, mirrors, beam splitters,etc.). In the example depicted in FIG. 2A, the bi-static configurationincludes a configuration where the outbound light and the inbound lightpass through a single optical window 124 but scanning unit 104 includestwo light deflectors, a first light deflector 114A for outbound lightand a second light deflector 114B for inbound light (the inbound lightin LIDAR system includes emitted light reflected from objects in thescene, and may also include ambient light arriving from other sources).In the examples depicted in FIGS. 2E and 2G, the bi-static configurationincludes a configuration where the outbound light passes through a firstoptical window 124A, and the inbound light passes through a secondoptical window 124B. In all the example configurations above, theinbound and outbound optical paths differ from one another.

In this embodiment, all the components of LIDAR system 100 may becontained within a single housing 200, or may be divided among aplurality of housings. As shown, projecting unit 102 is associated witha single light source 112 that includes a laser diode 202A (or one ormore laser diodes coupled together) configured to emit light (projectedlight 204). In one non-limiting example, the light projected by lightsource 112 may be at a wavelength between about 800 nm and 950 nm, havean average power between about 50 mW and about 500 mW, have a peak powerbetween about 50 W and about 200 W, and a pulse width of between about 2ns and about 100 ns. In addition, light source 112 may optionally beassociated with optical assembly 202B used for manipulation of the lightemitted by laser diode 202A (e.g. for collimation, focusing, etc.). Itis noted that other types of light sources 112 may be used, and that thedisclosure is not restricted to laser diodes. In addition, light source112 may emit its light in different formats, such as light pulses,frequency modulated, continuous wave (CW), quasi-CW, or any other formcorresponding to the particular light source employed. The projectionformat and other parameters may be changed by the light source from timeto time based on different factors, such as instructions from processingunit 108. The projected light is projected towards an outbound deflector114A that functions as a steering element for directing the projectedlight in field of view 120. In this example, scanning unit 104 alsoinclude a pivotable return deflector 114B that direct photons (reflectedlight 206) reflected back from an object 208 within field of view 120toward sensor 116. The reflected light is detected by sensor 116 andinformation about the object (e.g., the distance to object 212) isdetermined by processing unit 108.

In this figure, LIDAR system 100 is connected to a host 210. Consistentwith the present disclosure, the term “host” refers to any computingenvironment that may interface with LIDAR system 100, it may be avehicle system (e.g., part of vehicle 110), a testing system, a securitysystem, a surveillance system, a traffic control system, an urbanmodelling system, or any system that monitors its surroundings. Suchcomputing environment may include at least one processor and/or may beconnected LIDAR system 100 via the cloud. In some embodiments, host 210may also include interfaces to external devices such as camera andsensors configured to measure different characteristics of host 210(e.g., acceleration, steering wheel deflection, reverse drive, etc.).Consistent with the present disclosure, LIDAR system 100 may be fixed toa stationary object associated with host 210 (e.g. a building, a tripod)or to a portable system associated with host 210 (e.g., a portablecomputer, a movie camera). Consistent with the present disclosure, LIDARsystem 100 may be connected to host 210, to provide outputs of LIDARsystem 100 (e.g., a 3D model, a reflectivity image) to host 210.Specifically, host 210 may use LIDAR system 100 to aid in detecting andscanning the environment of host 210 or any other environment. Inaddition, host 210 may integrate, synchronize or otherwise use togetherthe outputs of LIDAR system 100 with outputs of other sensing systems(e.g. cameras, microphones, radar systems). In one example, LIDAR system100 may be used by a security system. This embodiment is described ingreater detail below with reference to FIG. 7.

LIDAR system 100 may also include a bus 212 (or other communicationmechanisms) that interconnect subsystems and components for transferringinformation within LIDAR system 100. Optionally, bus 212 (or anothercommunication mechanism) may be used for interconnecting LIDAR system100 with host 210. In the example of FIG. 2A, processing unit 108includes two processors 118 to regulate the operation of projecting unit102, scanning unit 104, and sensing unit 106 in a coordinated mannerbased, at least partially, on information received from internalfeedback of LIDAR system 100. In other words, processing unit 108 may beconfigured to dynamically operate LIDAR system 100 in a closed loop. Aclosed loop system is characterized by having feedback from at least oneof the elements and updating one or more parameters based on thereceived feedback. Moreover, a closed loop system may receive feedbackand update its own operation, at least partially, based on thatfeedback. A dynamic system or element is one that may be updated duringoperation.

According to some embodiments, scanning the environment around LIDARsystem 100 may include illuminating field of view 120 with light pulses.The light pulses may have parameters such as: pulse duration, pulseangular dispersion, wavelength, instantaneous power, photon density atdifferent distances from light source 112, average power, pulse powerintensity, pulse width, pulse repetition rate, pulse sequence, pulseduty cycle, wavelength, phase, polarization, and more. Scanning theenvironment around LIDAR system 100 may also include detecting andcharacterizing various aspects of the reflected light. Characteristicsof the reflected light may include, for example: time-of-flight (i.e.,time from emission until detection), instantaneous power (e.g., powersignature), average power across entire return pulse, and photondistribution/signal over return pulse period. By comparingcharacteristics of a light pulse with characteristics of correspondingreflections, a distance and possibly a physical characteristic, such asreflected intensity of object 212 may be estimated. By repeating thisprocess across multiple adjacent portions 122, in a predefined pattern(e.g., raster, Lissajous or other patterns) an entire scan of field ofview 120 may be achieved. As discussed below in greater detail, in somesituations LIDAR system 100 may direct light to only some of theportions 122 in field of view 120 at every scanning cycle. Theseportions may be adjacent to each other, but not necessarily so.

In another embodiment, LIDAR system 100 may include network interface214 for communicating with host 210 (e.g., a vehicle controller). Thecommunication between LIDAR system 100 and host 210 is represented by adashed arrow. In one embodiment, network interface 214 may include anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, networkinterface 214 may include a local area network (LAN) card to provide adata communication connection to a compatible LAN. In anotherembodiment, network interface 214 may include an Ethernet port connectedto radio frequency receivers and transmitters and/or optical (e.g.,infrared) receivers and transmitters. The specific design andimplementation of network interface 214 depends on the communicationsnetwork(s) over which LIDAR system 100 and host 210 are intended tooperate. For example, network interface 214 may be used, for example, toprovide outputs of LIDAR system 100 to the external system, such as a 3Dmodel, operational parameters of LIDAR system 100, and so on. In otherembodiment, the communication unit may be used, for example, to receiveinstructions from the external system, to receive information regardingthe inspected environment, to receive information from another sensor,etc.

FIG. 2B illustrates an example of a monostatic configuration of LIDARsystem 100 including a plurality projecting units 102. The term“monostatic configuration” broadly refers to LIDAR system configurationsin which the projected light exiting from the LIDAR system and thereflected light entering the LIDAR system pass through substantiallysimilar optical paths. In one example, the outbound light beam and theinbound light beam may share at least one optical assembly through whichboth outbound and inbound light beams pass. In another example, theoutbound light may pass through an optical window (not shown) and theinbound light radiation may pass through the same optical window. Amonostatic configuration may include a configuration where the scanningunit 104 includes a single light deflector 114 that directs theprojected light towards field of view 120 and directs the reflectedlight towards a sensor 116. As shown, both projected light 204 andreflected light 206 hits an asymmetrical deflector 216. The term“asymmetrical deflector” refers to any optical device having two sidescapable of deflecting a beam of light hitting it from one side in adifferent direction than it deflects a beam of light hitting it from thesecond side. In one example, the asymmetrical deflector does not deflectprojected light 204 and deflects reflected light 206 towards sensor 116.One example of an asymmetrical deflector may include a polarization beamsplitter. In another example, asymmetrical 216 may include an opticalisolator that allows the passage of light in only one direction. Adiagrammatic representation of asymmetrical deflector 216 is illustratedin FIG. 2D. Consistent with the present disclosure, a monostaticconfiguration of LIDAR system 100 may include an asymmetrical deflectorto prevent reflected light from hitting light source 112, and to directall the reflected light toward sensor 116, thereby increasing detectionsensitivity.

In the embodiment of FIG. 2B, LIDAR system 100 includes three projectingunits 102 each with a single of light source 112 aimed at a common lightdeflector 114. In one embodiment, the plurality of light sources 112(including two or more light sources) may project light withsubstantially the same wavelength and each light source 112 is generallyassociated with a differing area of the field of view (denoted in thefigure as 120A, 120B, and 120C). This enables scanning of a broaderfield of view than can be achieved with a light source 112. In anotherembodiment, the plurality of light sources 102 may project light withdiffering wavelengths, and all the light sources 112 may be directed tothe same portion (or overlapping portions) of field of view 120.

FIG. 2C illustrates an example of LIDAR system 100 in which projectingunit 102 includes a primary light source 112A and a secondary lightsource 112B. Primary light source 112A may project light with a longerwavelength than is sensitive to the human eye in order to optimize SNRand detection range. For example, primary light source 112A may projectlight with a wavelength between about 750 nm and 1100 nm. In contrast,secondary light source 112B may project light with a wavelength visibleto the human eye. For example, secondary light source 112B may projectlight with a wavelength between about 400 nm and 700 nm. In oneembodiment, secondary light source 112B may project light alongsubstantially the same optical path the as light projected by primarylight source 112A. Both light sources may be time-synchronized and mayproject light emission together or in interleaved pattern. An interleavepattern means that the light sources are not active at the same timewhich may mitigate mutual interference. A person who is of skill in theart would readily see that other combinations of wavelength ranges andactivation schedules may also be implemented.

Consistent with some embodiments, secondary light source 112B may causehuman eyes to blink when it is too close to the LIDAR optical outputport. This may ensure an eye safety mechanism not feasible with typicallaser sources that utilize the near-infrared light spectrum. In anotherembodiment, secondary light source 112B may be used for calibration andreliability at a point of service, in a manner somewhat similar to thecalibration of headlights with a special reflector/pattern at a certainheight from the ground with respect to vehicle 110. An operator at apoint of service could examine the calibration of the LIDAR by simplevisual inspection of the scanned pattern over a featured target such atest pattern board at a designated distance from LIDAR system 100. Inaddition, secondary light source 112B may provide means for operationalconfidence that the LIDAR is working for the end-user. For example, thesystem may be configured to permit a human to place a hand in front oflight deflector 114 to test its operation.

Secondary light source 112B may also have a non-visible element that candouble as a backup system in case primary light source 112A fails. Thisfeature may be useful for fail-safe devices with elevated functionalsafety ratings. Given that secondary light source 112B may be visibleand also due to reasons of cost and complexity, secondary light source112B may be associated with a smaller power compared to primary lightsource 112A. Therefore, in case of a failure of primary light source112A, the system functionality will fall back to secondary light source112B set of functionalities and capabilities. While the capabilities ofsecondary light source 112B may be inferior to the capabilities ofprimary light source 112A, LIDAR system 100 system may be designed insuch a fashion to enable vehicle 110 to safely arrive its destination.

FIG. 2D illustrates asymmetrical deflector 216 that may be part of LIDARsystem 100. In the illustrated example, asymmetrical deflector 216includes a reflective surface 218 (such as a mirror) and a one-waydeflector 220. While not necessarily so, asymmetrical deflector 216 mayoptionally be a static deflector. Asymmetrical deflector 216 may be usedin a monostatic configuration of LIDAR system 100, in order to allow acommon optical path for transmission and for reception of light via theat least one deflector 114, e.g. as illustrated in FIGS. 2B and 2C.However, typical asymmetrical deflectors such as beam splitters arecharacterized by energy losses, especially in the reception path, whichmay be more sensitive to power loses than the transmission path.

As depicted in FIG. 2D, LIDAR system 100 may include asymmetricaldeflector 216 positioned in the transmission path, which includesone-way deflector 220 for separating between the transmitted andreceived light signals. Optionally, one-way deflector 220 may besubstantially transparent to the transmission light and substantiallyreflective to the received light. The transmitted light is generated byprojecting unit 102 and may travel through one-way deflector 220 toscanning unit 104 which deflects it towards the optical outlet. Thereceived light arrives through the optical inlet, to the at least onedeflecting element 114, which deflects the reflections signal into aseparate path away from the light source and towards sensing unit 106.Optionally, asymmetrical deflector 216 may be combined with a polarizedlight source 112 which is linearly polarized with the same polarizationaxis as one-way deflector 220. Notably, the cross-section of theoutbound light beam is much smaller than that of the reflectionssignals. Accordingly, LIDAR system 100 may include one or more opticalcomponents (e.g. lens, collimator) for focusing or otherwisemanipulating the emitted polarized light beam to the dimensions of theasymmetrical deflector 216. In one embodiment, one-way deflector 220 maybe a polarizing beam splitter that is virtually transparent to thepolarized light beam.

Consistent with some embodiments, LIDAR system 100 may further includeoptics 222 (e.g., a quarter wave plate retarder) for modifying apolarization of the emitted light. For example, optics 222 may modify alinear polarization of the emitted light beam to circular polarization.Light reflected back to system 100 from the field of view would arriveback through deflector 114 to optics 222, bearing a circularpolarization with a reversed handedness with respect to the transmittedlight. Optics 222 would then convert the received reversed handednesspolarization light to a linear polarization that is not on the same axisas that of the polarized beam splitter 216. As noted above, the receivedlight-patch is larger than the transmitted light-patch, due to opticaldispersion of the beam traversing through the distance to the target.

Some of the received light will impinge on one-way deflector 220 thatwill reflect the light towards sensor 106 with some power loss. However,another part of the received patch of light will fall on a reflectivesurface 218 which surrounds one-way deflector 220 (e.g., polarizing beamsplitter slit). Reflective surface 218 will reflect the light towardssensing unit 106 with substantially zero power loss. One-way deflector220 would reflect light that is composed of various polarization axesand directions that will eventually arrive at the detector. Optionally,sensing unit 106 may include sensor 116 that is agnostic to the laserpolarization, and is primarily sensitive to the amount of impingingphotons at a certain wavelength range.

It is noted that the proposed asymmetrical deflector 216 provides farsuperior performances when compared to a simple mirror with a passagehole in it. In a mirror with a hole, all of the reflected light whichreaches the hole is lost to the detector. However, in deflector 216,one-way deflector 220 deflects a significant portion of that light(e.g., about 50%) toward the respective sensor 116. In LIDAR systems,the number photons reaching the LIDAR from remote distances is verylimited, and therefore the improvement in photon capture rate isimportant.

According to some embodiments, a device for beam splitting and steeringis described. A polarized beam may be emitted from a light source havinga first polarization. The emitted beam may be directed to pass through apolarized beam splitter assembly. The polarized beam splitter assemblyincludes on a first side a one-directional slit and on an opposing sidea mirror. The one-directional slit enables the polarized emitted beam totravel toward a quarter-wave-plate/wave-retarder which changes theemitted signal from a polarized signal to a linear signal (or viceversa) so that subsequently reflected beams cannot travel through theone-directional slit.

FIG. 2E shows an example of a bi-static configuration of LIDAR system100 without scanning unit 104. In order to illuminate an entire field ofview (or substantially the entire field of view) without deflector 114,projecting unit 102 may optionally include an array of light sources(e.g., 112A-112F). In one embodiment, the array of light sources mayinclude a linear array of light sources controlled by processor 118. Forexample, processor 118 may cause the linear array of light sources tosequentially project collimated laser beams towards first optionaloptical window 124A. First optional optical window 124A may include adiffuser lens for spreading the projected light and sequentially formingwide horizontal and narrow vertical beams. Optionally, some or all ofthe at least one light source 112 of system 100 may project lightconcurrently. For example, processor 118 may cause the array of lightsources to simultaneously project light beams from a plurality ofnon-adjacent light sources 112. In the depicted example, light source112A, light source 112D, and light source 112F simultaneously projectlaser beams towards first optional optical window 124A therebyilluminating the field of view with three narrow vertical beams. Thelight beam from fourth light source 112D may reach an object in thefield of view. The light reflected from the object may be captured bysecond optical window 124B and may be redirected to sensor 116. Theconfiguration depicted in FIG. 2E is considered to be a bi-staticconfiguration because the optical paths of the projected light and thereflected light are substantially different. It is noted that projectingunit 102 may also include a plurality of light sources 112 arranged innon-linear configurations, such as a two dimensional array, in hexagonaltiling, or in any other way.

FIG. 2F illustrates an example of a monostatic configuration of LIDARsystem 100 without scanning unit 104. Similar to the example embodimentrepresented in FIG. 2E, in order to illuminate an entire field of viewwithout deflector 114, projecting unit 102 may include an array of lightsources (e.g., 112A-112F). But, in contrast to FIG. 2E, thisconfiguration of LIDAR system 100 may include a single optical window124 for both the projected light and for the reflected light. Usingasymmetrical deflector 216, the reflected light may be redirected tosensor 116. The configuration depicted in FIG. 2E is considered to be amonostatic configuration because the optical paths of the projectedlight and the reflected light are substantially similar to one another.The term “substantially similar” in the context of the optical paths ofthe projected light and the reflected light means that the overlapbetween the two optical paths may be more than 80%, more than 85%, morethan 90%, or more than 95%.

FIG. 2G illustrates an example of a bi-static configuration of LIDARsystem 100. The configuration of LIDAR system 100 in this figure issimilar to the configuration shown in FIG. 2A. For example, bothconfigurations include a scanning unit 104 for directing projected lightin the outbound direction toward the field of view. But, in contrast tothe embodiment of FIG. 2A, in this configuration, scanning unit 104 doesnot redirect the reflected light in the inbound direction. Instead thereflected light passes through second optical window 124B and enterssensor 116. The configuration depicted in FIG. 2G is considered to be abi-static configuration because the optical paths of the projected lightand the reflected light are substantially different from one another.The term “substantially different” in the context of the optical pathsof the projected light and the reflected light means that the overlapbetween the two optical paths may be less than 10%, less than 5%, lessthan 1%, or less than 0.25%.

The Scanning Unit

FIGS. 3A-3D depict various configurations of scanning unit 104 and itsrole in LIDAR system 100. Specifically, FIG. 3A is a diagramillustrating scanning unit 104 with a MEMS mirror (e.g., square shaped),FIG. 3B is a diagram illustrating another scanning unit 104 with a MEMSmirror (e.g., round shaped), FIG. 3C is a diagram illustrating scanningunit 104 with an array of reflectors used for monostatic scanning LIDARsystem, and FIG. 3D is a diagram illustrating an example LIDAR system100 that mechanically scans the environment around LIDAR system 100. Oneskilled in the art will appreciate that the depicted configurations ofscanning unit 104 are exemplary only and may have numerous variationsand modifications within the scope of this disclosure.

FIG. 3A illustrates an example scanning unit 104 with a single axissquare MEMS mirror 300. In this example MEMS mirror 300 functions as atleast one deflector 114. As shown, scanning unit 104 may include one ormore actuators 302 (specifically, 302A and 302B). In one embodiment,actuator 302 may be made of semiconductor (e.g., silicon) and includes apiezoelectric layer (e.g. PZT, Lead zirconate titanate, aluminumnitride), which changes its dimension in response to electric signalsapplied by an actuation controller, a semi conductive layer, and a baselayer. In one embodiment, the physical properties of actuator 302 maydetermine the mechanical stresses that actuator 302 experiences whenelectrical current passes through it. When the piezoelectric material isactivated it exerts force on actuator 302 and causes it to bend. In oneembodiment, the resistivity of one or more actuators 302 may be measuredin an active state (Ractive) when mirror 300 is deflected at a certainangular position and compared to the resistivity at a resting state(Rrest). Feedback including Ractive may provide information to determinethe actual mirror deflection angle compared to an expected angle, and,if needed, mirror 300 deflection may be corrected. The differencebetween Rrest and Ractive may be correlated by a mirror drive into anangular deflection value that may serve to close the loop. Thisembodiment may be used for dynamic tracking of the actual mirrorposition and may optimize response, amplitude, deflection efficiency,and frequency for both linear mode and resonant mode MEMS mirrorschemes. This embodiment is described in greater detail below withreference to FIGS. 32-34.

During scanning, current (represented in the figure as the dashed line)may flow from contact 304A to contact 304B (through actuator 302A,spring 306A, mirror 300, spring 306B, and actuator 302B). Isolation gapsin semiconducting frame 308 such as isolation gap 310 may cause actuator302A and 302B to be two separate islands connected electrically throughsprings 306 and frame 308. The current flow, or any associatedelectrical parameter (voltage, current frequency, capacitance, relativedielectric constant, etc.), may be monitored by an associated positionfeedback. In case of a mechanical failure—where one of the components isdamaged—the current flow through the structure would alter and changefrom its functional calibrated values. At an extreme situation (forexample, when a spring is broken), the current would stop completely dueto a circuit break in the electrical chain by means of a faulty element.

FIG. 3B illustrates another example scanning unit 104 with a dual axisround MEMS mirror 300. In this example MEMS mirror 300 functions as atleast one deflector 114. In one embodiment, MEMS mirror 300 may have adiameter of between about 1 mm to about 5 mm. As shown, scanning unit104 may include four actuators 302 (302A, 302B, 302C, and 302D) each maybe at a differing length. In the illustrated example, the current(represented in the figure as the dashed line) flows from contact 304Ato contact 304D, but in other cases current may flow from contact 304Ato contact 304B, from contact 304A to contact 304C, from contact 304B tocontact 304C, from contact 304B to contact 304D, or from contact 304C tocontact 304D. Consistent with some embodiments, a dual axis MEMS mirrormay be configured to deflect light in a horizontal direction and in avertical direction. For example, the angles of deflection of a dual axisMEMS mirror may be between about 0° to 30° in the vertical direction andbetween about 0° to 50° in the horizontal direction. One skilled in theart will appreciate that the depicted configuration of mirror 300 mayhave numerous variations and modifications. In one example, at least ofdeflector 114 may have a dual axis square-shaped mirror or single axisround-shaped mirror. Examples of round and square mirror are depicted inFIGS. 3A and 3B as examples only. Any shape may be employed depending onsystem specifications. In one embodiment, actuators 302 may beincorporated as an integral part of at least of deflector 114, such thatpower to move MEMS mirror 300 is applied directly towards it. Inaddition, MEMS mirror 300 maybe connected to frame 308 by one or morerigid supporting elements. In another embodiment, at least of deflector114 may include an electrostatic or electromagnetic MEMS mirror.

As described above, a monostatic scanning LIDAR system utilizes at leasta portion of the same optical path for emitting projected light 204 andfor receiving reflected light 206. The light beam in the outbound pathmay be collimated and focused into a narrow beam while the reflectionsin the return path spread into a larger patch of light, due todispersion. In one embodiment, scanning unit 104 may have a largereflection area in the return path and asymmetrical deflector 216 thatredirects the reflections (i.e., reflected light 206) to sensor 116. Inone embodiment, scanning unit 104 may include a MEMS mirror with a largereflection area and negligible impact on the field of view and the framerate performance Additional details about the asymmetrical deflector 216are provided below with reference to FIG. 2D.

In some embodiments (e.g. as exemplified in FIG. 3C), scanning unit 104may include a deflector array (e.g. a reflector array) with small lightdeflectors (e.g. mirrors). In one embodiment, implementing lightdeflector 114 as a group of smaller individual light deflectors workingin synchronization may allow light deflector 114 to perform at a highscan rate with larger angles of deflection. The deflector array mayessentially act as a large light deflector (e.g. a large mirror) interms of effective area. The deflector array may be operated using ashared steering assembly configuration that allows sensor 116 to collectreflected photons from substantially the same portion of field of view120 being concurrently illuminated by light source 112. The term“concurrently” means that the two selected functions occur duringcoincident or overlapping time periods, either where one begins and endsduring the duration of the other, or where a later one starts before thecompletion of the other.

FIG. 3C illustrates an example of scanning unit 104 with a reflectorarray 312 having small mirrors. In this embodiment, reflector array 312functions as at least one deflector 114. Reflector array 312 may includea plurality of reflector units 314 configured to pivot (individually ortogether) and steer light pulses toward field of view 120. For example,reflector array 312 may be a part of an outbound path of light projectedfrom light source 112. Specifically, reflector array 312 may directprojected light 204 towards a portion of field of view 120. Reflectorarray 312 may also be part of a return path for light reflected from asurface of an object located within an illumined portion of field ofview 120. Specifically, reflector array 312 may direct reflected light206 towards sensor 116 or towards asymmetrical deflector 216. In oneexample, the area of reflector array 312 may be between about 75 toabout 150 mm², where each reflector units 314 may have a width of about10 μm and the supporting structure may be lower than 100 μm.

According to some embodiments, reflector array 312 may include one ormore sub-groups of steerable deflectors. Each sub-group of electricallysteerable deflectors may include one or more deflector units, such asreflector unit 314. For example, each steerable deflector unit 314 mayinclude at least one of a MEMS mirror, a reflective surface assembly,and an electromechanical actuator. In one embodiment, each reflectorunit 314 may be individually controlled by an individual processor (notshown), such that it may tilt towards a specific angle along each of oneor two separate axes. Alternatively, reflector array 312 may beassociated with a common controller (e.g., processor 118) configured tosynchronously manage the movement of reflector units 314 such that atleast part of them will pivot concurrently and point in approximatelythe same direction.

In addition, at least one processor 118 may select at least onereflector unit 314 for the outbound path (referred to hereinafter as “TXMirror”) and a group of reflector units 314 for the return path(referred to hereinafter as “RX Mirror”). Consistent with the presentdisclosure, increasing the number of TX Mirrors may increase a reflectedphotons beam spread. Additionally, decreasing the number of RX Mirrorsmay narrow the reception field and compensate for ambient lightconditions (such as clouds, rain, fog, extreme heat, and otherenvironmental conditions) and improve the signal to noise ratio. Also,as indicated above, the emitted light beam is typically narrower thanthe patch of reflected light, and therefore can be fully deflected by asmall portion of the deflection array. Moreover, it is possible to blocklight reflected from the portion of the deflection array used fortransmission (e.g. the TX mirror) from reaching sensor 116, therebyreducing an effect of internal reflections of the LIDAR system 100 onsystem operation. In addition, at least one processor 118 may pivot oneor more reflector units 314 to overcome mechanical impairments anddrifts due, for example, to thermal and gain effects. In an example, oneor more reflector units 314 may move differently than intended(frequency, rate, speed etc.) and their movement may be compensated forby electrically controlling the deflectors appropriately.

FIG. 3D illustrates an exemplary LIDAR system 100 that mechanicallyscans the environment of LIDAR system 100. In this example, LIDAR system100 may include a motor or other mechanisms for rotating housing 200about the axis of the LIDAR system 100. Alternatively, the motor (orother mechanism) may mechanically rotate a rigid structure of LIDARsystem 100 on which one or more light sources 112 and one or moresensors 116 are installed, thereby scanning the environment. Asdescribed above, projecting unit 102 may include at least one lightsource 112 configured to project light emission. The projected lightemission may travel along an outbound path towards field of view 120.Specifically, the projected light emission may be reflected by deflector114A through an exit aperture 314 when projected light 204 traveltowards optional optical window 124. The reflected light emission maytravel along a return path from object 208 towards sensing unit 106. Forexample, the reflected light 206 may be reflected by deflector 114B whenreflected light 206 travels towards sensing unit 106. A person skilledin the art would appreciate that a LIDAR system with a rotationmechanism for synchronically rotating one or more light sources or oneor more sensors, may use this synchronized rotation instead of (or inaddition to) steering an internal light deflector.

In embodiments in which the scanning of field of view 120 is mechanical,the projected light emission may be directed to exit aperture 314 thatis part of a wall 316 separating projecting unit 102 from other parts ofLIDAR system 100. In some examples, wall 316 can be formed from atransparent material (e.g., glass) coated with a reflective material toform deflector 114B. In this example, exit aperture 314 may correspondto the portion of wall 316 that is not coated by the reflectivematerial. Additionally or alternatively, exit aperture 314 may include ahole or cut-away in the wall 316. Reflected light 206 may be reflectedby deflector 114B and directed towards an entrance aperture 318 ofsensing unit 106. In some examples, an entrance aperture 318 may includea filtering window configured to allow wavelengths in a certainwavelength range to enter sensing unit 106 and attenuate otherwavelengths. The reflections of object 208 from field of view 120 may bereflected by deflector 114B and hit sensor 116. By comparing severalproperties of reflected light 206 with projected light 204, at least oneaspect of object 208 may be determined. For example, by comparing a timewhen projected light 204 was emitted by light source 112 and a time whensensor 116 received reflected light 206, a distance between object 208and LIDAR system 100 may be determined. In some examples, other aspectsof object 208, such as shape, color, material, etc. may also bedetermined.

In some examples, the LIDAR system 100 (or part thereof, including atleast one light source 112 and at least one sensor 116) may be rotatedabout at least one axis to determine a three-dimensional map of thesurroundings of the LIDAR system 100. For example, the LIDAR system 100may be rotated about a substantially vertical axis as illustrated byarrow 320 in order to scan field of 120. Although FIG. 3D illustratesthat the LIDAR system 100 is rotated clock-wise about the axis asillustrated by the arrow 320, additionally or alternatively, the LIDARsystem 100 may be rotated in a counter clockwise direction. In someexamples, the LIDAR system 100 may be rotated 360 degrees about thevertical axis. In other examples, the LIDAR system 100 may be rotatedback and forth along a sector smaller than 360-degree of the LIDARsystem 100. For example, the LIDAR system 100 may be mounted on aplatform that wobbles back and forth about the axis without making acomplete rotation.

The Sensing Unit

FIGS. 4A-4E depict various configurations of sensing unit 106 and itsrole in LIDAR system 100. Specifically, FIG. 4A is a diagramillustrating an example sensing unit 106 with a detector array, FIG. 4Bis a diagram illustrating monostatic scanning using a two-dimensionalsensor, FIG. 4C is a diagram illustrating an example of atwo-dimensional sensor 116, FIG. 4D is a diagram illustrating a lensarray associated with sensor 116, and FIG. 4E includes three diagramsillustrating the lens structure. One skilled in the art will appreciatethat the depicted configurations of sensing unit 106 are exemplary onlyand may have numerous alternative variations and modificationsconsistent with the principles of this disclosure.

FIG. 4A illustrates an example of sensing unit 106 with detector array400. In this example, at least one sensor 116 includes detector array400. LIDAR system 100 is configured to detect objects (e.g., bicycle208A and cloud 208B) in field of view 120 located at different distancesfrom LIDAR system 100 (could be meters or more). Objects 208 may be asolid object (e.g. a road, a tree, a car, a person), fluid object (e.g.fog, water, atmosphere particles), or object of another type (e.g. dustor a powdery illuminated object). When the photons emitted from lightsource 112 hit object 208 they either reflect, refract, or get absorbed.Typically, as shown in the figure, only a portion of the photonsreflected from object 208A enters optional optical window 124. As each˜15 cm change in distance results in a travel time difference of 1 ns(since the photons travel at the speed of light to and from object 208),the time differences between the travel times of different photonshitting the different objects may be detectable by a time-of-flightsensor with sufficiently quick response.

Sensor 116 includes a plurality of detection elements 402 for detectingphotons of a photonic pulse reflected back from field of view 120. Thedetection elements may all be included in detector array 400, which mayhave a rectangular arrangement (e.g. as shown) or any other arrangement.Detection elements 402 may operate concurrently or partiallyconcurrently with each other. Specifically, each detection element 402may issue detection information for every sampling duration (e.g. every1 nanosecond). In one example, detector array 400 may be a SiPM (Siliconphotomultipliers) which is a solid-state single-photon-sensitive devicebuilt from an array of single photon avalanche diode (, SPAD, serving asdetection elements 402) on a common silicon substrate. Similarphotomultipliers from other, non-silicon materials may also be used.Although a SiPM device works in digital/switching mode, the SiPM is ananalog device because all the microcells are read in parallel, making itpossible to generate signals within a dynamic range from a single photonto hundreds and thousands of photons detected by the different SPADs. Asmentioned above, more than one type of sensor may be implemented (e.g.SiPM and APD). Possibly, sensing unit 106 may include at least one APDintegrated into an SiPM array and/or at least one APD detector locatednext to a SiPM on a separate or common silicon substrate.

In one embodiment, detection elements 402 may be grouped into aplurality of regions 404. The regions are geometrical locations orenvironments within sensor 116 (e.g. within detector array 400)—and maybe shaped in different shapes (e.g. rectangular as shown, squares,rings, and so on, or in any other shape). While not all of theindividual detectors, which are included within the geometrical area ofa region 404, necessarily belong to that region, in most cases they willnot belong to other regions 404 covering other areas of the sensor310—unless some overlap is desired in the seams between regions. Asillustrated in FIG. 4A, the regions may be non-overlapping regions 404,but alternatively, they may overlap. Every region may be associated witha regional output circuitry 406 associated with that region. Theregional output circuitry 406 may provide a region output signal of acorresponding group of detection elements 402. For example, the regionof output circuitry 406 may be a summing circuit, but other forms ofcombined output of the individual detector into a unitary output(whether scalar, vector, or any other format) may be employed.Optionally, each region 404 is a single SiPM, but this is notnecessarily so, and a region may be a sub-portion of a single SiPM, agroup of several SiPMs, or even a combination of different types ofdetectors.

In the illustrated example, processing unit 108 is located at aseparated housing 200B (within or outside) host 210 (e.g. within vehicle110), and sensing unit 106 may include a dedicated processor 408 foranalyzing the reflected light. Alternatively, processing unit 108 may beused for analyzing reflected light 206. It is noted that LIDAR system100 may be implemented multiple housings in other ways than theillustrated example. For example, light deflector 114 may be located ina different housing than projecting unit 102 and/or sensing module 106.In one embodiment, LIDAR system 100 may include multiple housingsconnected to each other in different ways, such as: electric wireconnection, wireless connection (e.g., RF connection), fiber opticscable, and any combination of the above.

In one embodiment, analyzing reflected light 206 may include determininga time of flight for reflected light 206, based on outputs of individualdetectors of different regions. Optionally, processor 408 may beconfigured to determine the time of flight for reflected light 206 basedon the plurality of regions of output signals. In addition to the timeof flight, processing unit 108 may analyze reflected light 206 todetermine the average power across an entire return pulse, and thephoton distribution/signal may be determined over the return pulseperiod (“pulse shape”). In the illustrated example, the outputs of anydetection elements 402 may not be transmitted directly to processor 408,but rather combined (e.g. summed) with signals of other detectors of theregion 404 before being passed to processor 408. However, this is onlyan example and the circuitry of sensor 116 may transmit information froma detection element 402 to processor 408 via other routes (not via aregion output circuitry 406).

FIG. 4B is a diagram illustrating LIDAR system 100 configured to scanthe environment of LIDAR system 100 using a two-dimensional sensor 116.In the example of FIG. 4B, sensor 116 is a matrix of 4×6 detectors 410(also referred to as “pixels”). In one embodiment, a pixel size may beabout 1×1 mm. Sensor 116 is two-dimensional in the sense that it hasmore than one set (e.g. row, column) of detectors 410 in twonon-parallel axes (e.g. orthogonal axes, as exemplified in theillustrated examples). The number of detectors 410 in sensor 116 mayvary between differing implementations, e.g. depending on the desiredresolution, signal to noise ratio (SNR), desired detection distance, andso on. For example, sensor 116 may have anywhere between 5 and 5,000pixels. In another example (not shown in the figure) Also, sensor 116may be a one-dimensional matrix (e.g. 1×8 pixels).

It is noted that each detector 410 may include a plurality of detectionelements 402, such as Avalanche Photo Diodes (APD), Single PhotonAvalanche Diodes (SPADs), combination of Avalanche Photo Diodes (APD)and Single Photon Avalanche Diodes (SPADs) or detecting elements thatmeasure both the time of flight from a laser pulse transmission event tothe reception event and the intensity of the received photons. Forexample, each detector 410 may include anywhere between 20 and 5,000SPADs. The outputs of detection elements 402 in each detector 410 may besummed, averaged, or otherwise combined to provide a unified pixeloutput.

In the illustrated example, sensing unit 106 may include atwo-dimensional sensor 116 (or a plurality of two-dimensional sensors116), whose field of view is smaller than field of view 120 of LIDARsystem 100. In this discussion, field of view 120 (the overall field ofview which can be scanned by LIDAR system 100 without moving, rotatingor rolling in any direction) is denoted “first FOV 412”, and the smallerFOV of sensor 116 is denoted “second FOV 412” (interchangeably“instantaneous FOV”). The coverage area of second FOV 414 relative tothe first FOV 412 may differ, depending on the specific use of LIDARsystem 100, and may be, for example, between 0.5% and 50%. In oneexample, second FOV 412 may be between about 0.05° and 1° elongated inthe vertical dimension. Even if LIDAR system 100 includes more than onetwo-dimensional sensor 116, the combined field of view of the sensorsarray may still be smaller than the first FOV 412, e.g. by a factor ofat least 5, by a factor of at least 10, by a factor of at least 20, orby a factor of at least 50, for example.

In order to cover first FOV 412, scanning unit 106 may direct photonsarriving from different parts of the environment to sensor 116 atdifferent times. In the illustrated monostatic configuration, togetherwith directing projected light 204 towards field of view 120 and when atleast one light deflector 114 is located in an instantaneous position,scanning unit 106 may also direct reflected light 206 to sensor 116.Typically, at every moment during the scanning of first FOV 412, thelight beam emitted by LIDAR system 100 covers part of the environmentwhich is larger than the second FOV 414 (in angular opening) andincludes the part of the environment from which light is collected byscanning unit 104 and sensor 116.

FIG. 4C is a diagram illustrating an example of a two-dimensional sensor116. In this embodiment, sensor 116 is a matrix of 8×5 detectors 410 andeach detector 410 includes a plurality of detection elements 402. In oneexample, detector 410A is located in the second row (denoted “R2”) andthird column (denoted “C3”) of sensor 116, which includes a matrix of4×3 detection elements 402. In another example, detector 410B located inthe fourth row (denoted “R4”) and sixth column (denoted “C6”) of sensor116 includes a matrix of 3×3 detection elements 402. Accordingly, thenumber of detection elements 402 in each detector 410 may be constant,or may vary, and differing detectors 410 in a common array may have adifferent number of detection elements 402. The outputs of all detectionelements 402 in each detector 410 may be summed, averaged, or otherwisecombined to provide a single pixel-output value. It is noted that whiledetectors 410 in the example of FIG. 4C are arranged in a rectangularmatrix (straight rows and straight columns), other arrangements may alsobe used, e.g. a circular arrangement or a honeycomb arrangement.

According to some embodiments, measurements from each detector 410 mayenable determination of the time of flight from a light pulse emissionevent to the reception event and the intensity of the received photons.The reception event may be the result of the light pulse being reflectedfrom object 208. The time of flight may be a timestamp value thatrepresents the distance of the reflecting object to optional opticalwindow 124. Time of flight values may be realized by photon detectionand counting methods, such as Time Correlated Single Photon Counters(TCSPC), analog methods for photon detection such as signal integrationand qualification (via analog to digital converters or plaincomparators) or otherwise.

In some embodiments and with reference to FIG. 4B, during a scanningcycle, each instantaneous position of at least one light deflector 114may be associated with a particular portion 122 of field of view 120.The design of sensor 116 enables an association between the reflectedlight from a single portion of field of view 120 and multiple detectors410. Therefore, the scanning resolution of LIDAR system may berepresented by the number of instantaneous positions (per scanningcycle) times the number of detectors 410 in sensor 116. The informationfrom each detector 410 (i.e., each pixel) represents the basic dataelement that from which the captured field of view in thethree-dimensional space is built. This may include, for example, thebasic element of a point cloud representation, with a spatial positionand an associated reflected intensity value. In one embodiment, thereflections from a single portion of field of view 120 that are detectedby multiple detectors 410 may be returning from different objectslocated in the single portion of field of view 120. For example, thesingle portion of field of view 120 may be greater than 50×50 cm at thefar field, which can easily include two, three, or more objects partlycovered by each other.

FIG. 4D is a cross cut diagram of a part of sensor 116, in accordancewith examples of the presently disclosed subject matter. The illustratedpart of sensor 116 includes a part of a detector array 400 whichincludes four detection elements 402 (e.g., four SPADs, four APDs).Detector array 400 may be a photodetector sensor realized incomplementary metal-oxide-semiconductor (CMOS). Each of the detectionelements 402 has a sensitive area, which is positioned within asubstrate surrounding. While not necessarily so, sensor 116 may be usedin a monostatic LiDAR system having a narrow field of view (e.g.,because scanning unit 104 scans different parts of the field of view atdifferent times). The narrow field of view for the incoming lightbeam—if implemented—eliminates the problem of out-of-focus imaging. Asexemplified in FIG. 4D, sensor 116 may include a plurality of lenses 422(e.g., microlenses), each lens 422 may direct incident light toward adifferent detection element 402 (e.g., toward an active area ofdetection element 402), which may be usable when out-of-focus imaging isnot an issue. Lenses 422 may be used for increasing an optical fillfactor and sensitivity of detector array 400, because most of the lightthat reaches sensor 116 may be deflected toward the active areas ofdetection elements 402

Detector array 400, as exemplified in FIG. 4D, may include severallayers built into the silicon substrate by various methods (e.g.,implant) resulting in a sensitive area, contact elements to the metallayers and isolation elements (e.g., shallow trench implant STI, guardrings, optical trenches, etc.). The sensitive area may be a volumetricelement in the CMOS detector that enables the optical conversion ofincoming photons into a current flow given an adequate voltage bias isapplied to the device. In the case of a APD/SPAD, the sensitive areawould be a combination of an electrical field that pulls electronscreated by photon absorption towards a multiplication area where aphoton induced electron is amplified creating a breakdown avalanche ofmultiplied electrons.

A front side illuminated detector (e.g., as illustrated in FIG. 4D) hasthe input optical port at the same side as the metal layers residing ontop of the semiconductor (Silicon). The metal layers are required torealize the electrical connections of each individual photodetectorelement (e.g., anode and cathode) with various elements such as: biasvoltage, quenching/ballast elements, and other photodetectors in acommon array. The optical port through which the photons impinge uponthe detector sensitive area is comprised of a passage through the metallayer. It is noted that passage of light from some directions throughthis passage may be blocked by one or more metal layers (e.g., metallayer ML6, as illustrated for the leftmost detector elements 402 in FIG.4D). Such blockage reduces the total optical light absorbing efficiencyof the detector.

FIG. 4E illustrates three detection elements 402, each with anassociated lens 422, in accordance with examples of the presentingdisclosed subject matter. Each of the three detection elements of FIG.4E, denoted 402(1), 402(2), and 402(3), illustrates a lens configurationwhich may be implemented in associated with one or more of the detectingelements 402 of sensor 116. It is noted that combinations of these lensconfigurations may also be implemented.

In the lens configuration illustrated with regards to detection element402(1), a focal point of the associated lens 422 may be located abovethe semiconductor surface. Optionally, openings in different metallayers of the detection element may have different sizes aligned withthe cone of focusing light generated by the associated lens 422. Such astructure may improve the signal-to-noise and resolution of the array400 as a whole device. Large metal layers may be important for deliveryof power and ground shielding. This approach may be useful, e.g., with amonostatic LiDAR design with a narrow field of view where the incominglight beam is comprised of parallel rays and the imaging focus does nothave any consequence to the detected signal.

In the lens configuration illustrated with regards to detection element402(2), an efficiency of photon detection by the detection elements 402may be improved by identifying a sweet spot. Specifically, aphotodetector implemented in CMOS may have a sweet spot in the sensitivevolume area where the probability of a photon creating an avalancheeffect is the highest. Therefore, a focal point of lens 422 may bepositioned inside the sensitive volume area at the sweet spot location,as demonstrated by detection elements 402(2). The lens shape anddistance from the focal point may take into account the refractiveindices of all the elements the laser beam is passing along the way fromthe lens to the sensitive sweet spot location buried in thesemiconductor material.

In the lens configuration illustrated with regards to the detectionelement on the right of FIG. 4E, an efficiency of photon absorption inthe semiconductor material may be improved using a diffuser andreflective elements. Specifically, a near IR wavelength requires asignificantly long path of silicon material in order to achieve a highprobability of absorbing a photon that travels through. In a typicallens configuration, a photon may traverse the sensitive area and may notbe absorbed into a detectable electron. A long absorption path thatimproves the probability for a photon to create an electron renders thesize of the sensitive area towards less practical dimensions (tens of umfor example) for a CMOS device fabricated with typical foundryprocesses. The rightmost detector element in FIG. 4E demonstrates atechnique for processing incoming photons. The associated lens 422focuses the incoming light onto a diffuser element 424. In oneembodiment, light sensor 116 may further include a diffuser located inthe gap distant from the outer surface of at least some of thedetectors. For example, diffuser 424 may steer the light beam sideways(e.g., as perpendicular as possible) towards the sensitive area and thereflective optical trenches 426. The diffuser is located at the focalpoint, above the focal point, or below the focal point. In thisembodiment, the incoming light may be focused on a specific locationwhere a diffuser element is located. Optionally, detector element 422 isdesigned to optically avoid the inactive areas where a photon inducedelectron may get lost and reduce the effective detection efficiency.Reflective optical trenches 426 (or other forms of optically reflectivestructures) cause the photons to bounce back and forth across thesensitive area, thus increasing the likelihood of detection. Ideally,the photons will get trapped in a cavity consisting of the sensitivearea and the reflective trenches indefinitely until the photon isabsorbed and creates an electron/hole pair.

Consistent with the present disclosure, a long path is created for theimpinging photons to be absorbed and contribute to a higher probabilityof detection. Optical trenches may also be implemented in detectingelement 422 for reducing cross talk effects of parasitic photons createdduring an avalanche that may leak to other detectors and cause falsedetection events. According to some embodiments, a photo detector arraymay be optimized so that a higher yield of the received signal isutilized, meaning, that as much of the received signal is received andless of the signal is lost to internal degradation of the signal. Thephoto detector array may be improved by: (a) moving the focal point at alocation above the semiconductor surface, optionally by designing themetal layers above the substrate appropriately; (b) by steering thefocal point to the most responsive/sensitive area (or “sweet spot”) ofthe substrate and (c) adding a diffuser above the substrate to steer thesignal toward the “sweet spot” and/or adding reflective material to thetrenches so that deflected signals are reflected back to the “sweetspot.”

While in some lens configurations, lens 422 may be positioned so thatits focal point is above a center of the corresponding detection element402, it is noted that this is not necessarily so. In other lensconfiguration, a position of the focal point of the lens 422 withrespect to a center of the corresponding detection element 402 isshifted based on a distance of the respective detection element 402 froma center of the detection array 400. This may be useful in relativelylarger detection arrays 400, in which detector elements further from thecenter receive light in angles which are increasingly off-axis. Shiftingthe location of the focal points (e.g., toward the center of detectionarray 400) allows correcting for the incidence angles. Specifically,shifting the location of the focal points (e.g., toward the center ofdetection array 400) allows correcting for the incidence angles whileusing substantially identical lenses 422 for all detection elements,which are positioned at the same angle with respect to a surface of thedetector.

Adding an array of lenses 422 to an array of detection elements 402 maybe useful when using a relatively small sensor 116 which covers only asmall part of the field of view because in such a case, the reflectionsignals from the scene reach the detectors array 400 from substantiallythe same angle, and it is, therefore, easy to focus all the light ontoindividual detectors. It is also noted, that in one embodiment, lenses422 may be used in LIDAR system 100 for favoring about increasing theoverall probability of detection of the entire array 400 (preventingphotons from being “wasted” in the dead area betweendetectors/sub-detectors) at the expense of spatial distinctiveness. Thisembodiment is in contrast to prior art implementations such as CMOS RGBcamera, which prioritize spatial distinctiveness (i.e., light thatpropagates in the direction of detection element A is not allowed to bedirected by the lens toward detection element B, that is, to “bleed” toanother detection element of the array). Optionally, sensor 116 includesan array of lens 422, each being correlated to a corresponding detectionelement 402, while at least one of the lenses 422 deflects light whichpropagates to a first detection element 402 toward a second detectionelement 402 (thereby it may increase the overall probability ofdetection of the entire array).

Specifically, consistent with some embodiments of the presentdisclosure, light sensor 116 may include an array of light detectors(e.g., detector array 400), each light detector (e.g., detector 410)being configured to cause an electric current to flow when light passesthrough an outer surface of a respective detector. In addition, lightsensor 116 may include at least one micro-lens configured to directlight toward the array of light detectors, the at least one micro-lenshaving a focal point. Light sensor 116 may further include at least onelayer of conductive material interposed between the at least onemicro-lens and the array of light detectors and having a gap therein topermit light to pass from the at least one micro-lens to the array, theat least one layer being sized to maintain a space between the at leastone micro-lens and the array to cause the focal point (e.g., the focalpoint may be a plane) to be located in the gap, at a location spacedfrom the detecting surfaces of the array of light detectors.

In related embodiments, each detector may include a plurality of SinglePhoton Avalanche Diodes (SPADs) or a plurality of Avalanche Photo Diodes(APD). The conductive material may be a multi-layer metal constriction,and the at least one layer of conductive material may be electricallyconnected to detectors in the array. In one example, the at least onelayer of conductive material includes a plurality of layers. Inaddition, the gap may be shaped to converge from the at least onemicro-lens toward the focal point, and to diverge from a region of thefocal point toward the array. In other embodiments, light sensor 116 mayfurther include at least one reflector adjacent each photo detector. Inone embodiment, a plurality of micro-lenses may be arranged in a lensarray and the plurality of detectors may be arranged in a detectorarray. In another embodiment, the plurality of micro-lenses may includea single lens configured to project light to a plurality of detectors inthe array.

Referring by way of a nonlimiting example to FIGS. 2E, 2F and 2G, it isnoted that the one or more sensors 116 of system 100 may receive lightfrom a scanning deflector 114 or directly from the FOV without scanningEven if light from the entire FOV arrives to the at least one sensor 116at the same time, in some implementations the one or more sensors 116may sample only parts of the FOV for detection output at any given time.For example, if the illumination of projection unit 102 illuminatesdifferent parts of the FOV at different times (whether using a deflector114 and/or by activating different light sources 112 at differenttimes), light may arrive at all of the pixels or sensors 116 of sensingunit 106, and only pixels/sensors which are expected to detect the LIDARillumination may be actively collecting data for detection outputs. Thisway, the rest of the pixels/sensors do not unnecessarily collect ambientnoise. Referring to the scanning—in the outbound or in the inbounddirections—it is noted that substantially different scales of scanningmay be implemented. For example, in some implementations the scannedarea may cover 1‰ or 0.1‰ of the FOV, while in other implementations thescanned area may cover 10% or 25% of the FOV. All other relativeportions of the FOV values may also be implemented, of course.

The Processing Unit

FIGS. 5A-5C depict different functionalities of processing units 108 inaccordance with some embodiments of the present disclosure.Specifically, FIG. 5A is a diagram illustrating emission patterns in asingle frame-time for a single portion of the field of view, FIG. 5B isa diagram illustrating emission scheme in a single frame-time for thewhole field of view, and. FIG. 5C is a diagram illustrating the actuallight emission projected towards field of view during a single scanningcycle.

FIG. 5A illustrates four examples of emission patterns in a singleframe-time for a single portion 122 of field of view 120 associated withan instantaneous position of at least one light deflector 114.Consistent with embodiments of the present disclosure, processing unit108 may control at least one light source 112 and light deflector 114(or coordinate the operation of at least one light source 112 and atleast one light deflector 114) in a manner enabling light flux to varyover a scan of field of view 120. Consistent with other embodiments,processing unit 108 may control only at least one light source 112 andlight deflector 114 may be moved or pivoted in a fixed predefinedpattern.

Diagrams A-D in FIG. 5A depict the power of light emitted towards asingle portion 122 of field of view 120 over time. In Diagram A,processor 118 may control the operation of light source 112 in a mannersuch that during scanning of field of view 120 an initial light emissionis projected toward portion 122 of field of view 120. When projectingunit 102 includes a pulsed-light light source, the initial lightemission may include one or more initial pulses (also referred to as“pilot pulses”). Processing unit 108 may receive from sensor 116 pilotinformation about reflections associated with the initial lightemission. In one embodiment, the pilot information may be represented asa single signal based on the outputs of one or more detectors (e.g. oneor more SPADs, one or more APDs, one or more SiPMs, etc.) or as aplurality of signals based on the outputs of multiple detectors. In oneexample, the pilot information may include analog and/or digitalinformation. In another example, the pilot information may include asingle value and/or a plurality of values (e.g. for different timesand/or parts of the segment).

Based on information about reflections associated with the initial lightemission, processing unit 108 may be configured to determine the type ofsubsequent light emission to be projected towards portion 122 of fieldof view 120. The determined subsequent light emission for the particularportion of field of view 120 may be made during the same scanning cycle(i.e., in the same frame) or in a subsequent scanning cycle (i.e., in asubsequent frame). This embodiment is described in greater detail belowwith reference to FIGS. 23-25.

In Diagram B, processor 118 may control the operation of light source112 in a manner such that during scanning of field of view 120 lightpulses in different intensities are projected towards a single portion122 of field of view 120. In one embodiment, LIDAR system 100 may beoperable to generate depth maps of one or more different types, such asany one or more of the following types: point cloud model, polygon mesh,depth image (holding depth information for each pixel of an image or ofa 2D array), or any other type of 3D model of a scene. The sequence ofdepth maps may be a temporal sequence, in which different depth maps aregenerated at a different time. Each depth map of the sequence associatedwith a scanning cycle (interchangeably “frame”) may be generated withinthe duration of a corresponding subsequent frame-time. In one example, atypical frame-time may last less than a second. In some embodiments,LIDAR system 100 may have a fixed frame rate (e.g. 10 frames per second,25 frames per second, 50 frames per second) or the frame rate may bedynamic. In other embodiments, the frame-times of different frames maynot be identical across the sequence. For example, LIDAR system 100 mayimplement a 10 frames-per-second rate that includes generating a firstdepth map in 100 milliseconds (the average), a second frame in 92milliseconds, a third frame at 142 milliseconds, and so on.

In Diagram C, processor 118 may control the operation of light source112 in a manner such that during scanning of field of view 120 lightpulses associated with different durations are projected towards asingle portion 122 of field of view 120. In one embodiment, LIDAR system100 may be operable to generate a different number of pulses in eachframe. The number of pulses may vary between 0 to 32 pulses (e.g., 1, 5,12, 28, or more pulses) and may be based on information derived fromprevious emissions. The time between light pulses may depend on desireddetection range and can be between 500 ns and 5000 ns. In one example,processing unit 108 may receive from sensor 116 information aboutreflections associated with each light-pulse. Based on the information(or the lack of information), processing unit 108 may determine ifadditional light pulses are needed. It is noted that the durations ofthe processing times and the emission times in diagrams A-D are notin-scale. Specifically, the processing time may be substantially longerthan the emission time. In diagram D, projecting unit 102 may include acontinuous-wave light source. In one embodiment, the initial lightemission may include a period of time where light is emitted and thesubsequent emission may be a continuation of the initial emission, orthere may be a discontinuity. In one embodiment, the intensity of thecontinuous emission may change over time.

Consistent with some embodiments of the present disclosure, the emissionpattern may be determined per each portion of field of view 120. Inother words, processor 118 may control the emission of light to allowdifferentiation in the illumination of different portions of field ofview 120. In one example, processor 118 may determine the emissionpattern for a single portion 122 of field of view 120, based ondetection of reflected light from the same scanning cycle (e.g., theinitial emission), which makes LIDAR system 100 extremely dynamic. Inanother example, processor 118 may determine the emission pattern for asingle portion 122 of field of view 120, based on detection of reflectedlight from a previous scanning cycle. The differences in the patterns ofthe subsequent emissions may result from determining different valuesfor light-source parameters for the subsequent emission, such as any oneof the following.

-   -   a. Overall energy of the subsequent emission.    -   b. Energy profile of the subsequent emission.    -   c. A number of light-pulse-repetition per frame.    -   d. Light modulation characteristics such as duration, rate,        peak, average power, and pulse shape.    -   e. Wave properties of the subsequent emission, such as        polarization, wavelength, etc.

Consistent with the present disclosure, the differentiation in thesubsequent emissions may be put to different uses. In one example, it ispossible to limit emitted power levels in one portion of field of view120 where safety is a consideration, while emitting higher power levels(thus improving signal-to-noise ratio and detection range) for otherportions of field of view 120. This is relevant for eye safety, but mayalso be relevant for skin safety, safety of optical systems, safety ofsensitive materials, and more. In another example, it is possible todirect more energy towards portions of field of view 120 where it willbe of greater use (e.g. regions of interest, further distanced targets,low reflection targets, etc.) while limiting the lighting energy toother portions of field of view 120 based on detection results from thesame frame or previous frame. It is noted that processing unit 108 mayprocess detected signals from a single instantaneous field of viewseveral times within a single scanning frame time; for example,subsequent emission may be determined upon after every pulse emitted, orafter a number of pulses emitted.

FIG. 5B illustrates three examples of emission schemes in a singleframe-time for field of view 120. Consistent with embodiments of thepresent disclosure, at least on processing unit 108 may use obtainedinformation to dynamically adjust the operational mode of LIDAR system100 and/or determine values of parameters of specific components ofLIDAR system 100. The obtained information may be determined fromprocessing data captured in field of view 120, or received (directly orindirectly) from host 210. Processing unit 108 may use the obtainedinformation to determine a scanning scheme for scanning the differentportions of field of view 120. The obtained information may include acurrent light condition, a current weather condition, a current drivingenvironment of the host vehicle, a current location of the host vehicle,a current trajectory of the host vehicle, a current topography of roadsurrounding the host vehicle, or any other condition or objectdetectable through light reflection. In some embodiments, the determinedscanning scheme may include at least one of the following: (a) adesignation of portions within field of view 120 to be actively scannedas part of a scanning cycle, (b) a projecting plan for projecting unit102 that defines the light emission profile at different portions offield of view 120; (c) a deflecting plan for scanning unit 104 thatdefines, for example, a deflection direction, frequency, and designatingidle elements within a reflector array; and (d) a detection plan forsensing unit 106 that defines the detectors sensitivity or responsivitypattern.

In addition, processing unit 108 may determine the scanning scheme atleast partially by obtaining an identification of at least one region ofinterest within the field of view 120 and at least one region ofnon-interest within the field of view 120. In some embodiments,processing unit 108 may determine the scanning scheme at least partiallyby obtaining an identification of at least one region of high interestwithin the field of view 120 and at least one region of lower-interestwithin the field of view 120. The identification of the at least oneregion of interest within the field of view 120 may be determined, forexample, from processing data captured in field of view 120, based ondata of another sensor (e.g. camera, GPS), received (directly orindirectly) from host 210, or any combination of the above. In someembodiments, the identification of at least one region of interest mayinclude identification of portions, areas, sections, pixels, or objectswithin field of view 120 that are important to monitor. Examples ofareas that may be identified as regions of interest may include,crosswalks, moving objects, people, nearby vehicles or any otherenvironmental condition or object that may be helpful in vehiclenavigation. Examples of areas that may be identified as regions ofnon-interest (or lower-interest) may be static (non-moving) far-awaybuildings, a skyline, an area above the horizon and objects in the fieldof view. Upon obtaining the identification of at least one region ofinterest within the field of view 120, processing unit 108 may determinethe scanning scheme or change an existing scanning scheme. Further todetermining or changing the light-source parameters (as describedabove), processing unit 108 may allocate detector resources based on theidentification of the at least one region of interest. In one example,to reduce noise, processing unit 108 may activate detectors 410 where aregion of interest is expected and disable detectors 410 where regionsof non-interest are expected. In another example, processing unit 108may change the detector sensitivity, e.g., increasing sensor sensitivityfor long range detection where the reflected power is low.

Diagrams A-C in FIG. 5B depict examples of different scanning schemesfor scanning field of view 120. Each square in field of view 120represents a different portion 122 associated with an instantaneousposition of at least one light deflector 114. Legend 500 details thelevel of light flux represented by the filling pattern of the squares.Diagram A depicts a first scanning scheme in which all of the portionshave the same importance/priority and a default light flux is allocatedto them. The first scanning scheme may be utilized in a start-up phaseor periodically interleaved with another scanning scheme to monitor thewhole field of view for unexpected/new objects. In one example, thelight source parameters in the first scanning scheme may be configuredto generate light pulses at constant amplitudes. Diagram B depicts asecond scanning scheme in which a portion of field of view 120 isallocated with high light flux while the rest of field of view 120 isallocated with default light flux and low light flux. The portions offield of view 120 that are the least interesting may be allocated withlow light flux. Diagram C depicts a third scanning scheme in which acompact vehicle and a bus (see silhouettes) are identified in field ofview 120. In this scanning scheme, the edges of the vehicle and bus maybe tracked with high power and the central mass of the vehicle and busmay be allocated with less light flux (or no light flux). Such lightflux allocation enables concentration of more of the optical budget onthe edges of the identified objects and less on their center which haveless importance.

FIG. 5C illustrating the emission of light towards field of view 120during a single scanning cycle. In the depicted example, field of view120 is represented by an 8×9 matrix, where each of the 72 cellscorresponds to a separate portion 122 associated with a differentinstantaneous position of at least one light deflector 114. In thisexemplary scanning cycle, each portion includes one or more white dotsthat represent the number of light pulses projected toward that portion,and some portions include black dots that represent reflected light fromthat portion detected by sensor 116. As shown, field of view 120 isdivided into three sectors: sector I on the right side of field of view120, sector II in the middle of field of view 120, and sector III on theleft side of field of view 120. In this exemplary scanning cycle, sectorI was initially allocated with a single light pulse per portion; sectorII, previously identified as a region of interest, was initiallyallocated with three light pulses per portion; and sector III wasinitially allocated with two light pulses per portion. Also as shown,scanning of field of view 120 reveals four objects 208: two free-formobjects in the near field (e.g., between 5 and 50 meters), arounded-square object in the mid field (e.g., between 50 and 150meters), and a triangle object in the far field (e.g., between 150 and500 meters). While the discussion of FIG. 5C uses number of pulses as anexample of light flux allocation, it is noted that light flux allocationto different parts of the field of view may also be implemented in otherways such as: pulse duration, pulse angular dispersion, wavelength,instantaneous power, photon density at different distances from lightsource 112, average power, pulse power intensity, pulse width, pulserepetition rate, pulse sequence, pulse duty cycle, wavelength, phase,polarization, and more. The illustration of the light emission as asingle scanning cycle in FIG. 5C demonstrates different capabilities ofLIDAR system 100. In a first embodiment, processor 118 is configured touse two light pulses to detect a first object (e.g., the rounded-squareobject) at a first distance, and to use three light pulses to detect asecond object (e.g., the triangle object) at a second distance greaterthan the first distance. This embodiment is described in greater detailbelow with reference to FIGS. 11-13. In a second embodiment, processor118 is configured to allocate more light to portions of the field ofview where a region of interest is identified. Specifically, in thepresent example, sector II was identified as a region of interest andaccordingly it was allocated with three light pulses while the rest offield of view 120 was allocated with two or less light pulses. Thisembodiment is described in greater detail below with reference to FIGS.20-22. In a third embodiment, processor 118 is configured to controllight source 112 in a manner such that only a single light pulse isprojected toward to portions B1, B2, and C1 in FIG. 5C, although theyare part of sector III that was initially allocated with two lightpulses per portion. This occurs because the processing unit 108 detectedan object in the near field based on the first light pulse. Thisembodiment is described in greater detail below with reference to FIGS.23-25. Allocation of less than maximal amount of pulses may also be aresult of other considerations. For examples, in at least some regions,detection of object at a first distance (e.g. a near field object) mayresult in reducing an overall amount of light emitted to this portion offield of view 120. This embodiment is described in greater detail belowwith reference to FIGS. 14-16. Other reasons to for determining powerallocation to different portions is discussed below with respect toFIGS. 29-31, FIGS. 53-55, and FIGS. 50-52.

Additional details and examples on different components of LIDAR system100 and their associated functionalities are included in Applicant'sU.S. patent application Ser. No. 15/391,916 filed Dec. 28, 2016;Applicant's U.S. patent application Ser. No. 15/393,749 filed Dec. 29,2016; Applicant's U.S. patent application Ser. No. 15/393,285 filed Dec.29, 2016; and Applicant's U.S. patent application Ser. No. 15/393,593filed Dec. 29, 2016, which are incorporated herein by reference in theirentirety.

Example Implementation: Vehicle

FIGS. 6A-6C illustrate the implementation of LIDAR system 100 in avehicle (e.g., vehicle 110). Any of the aspects of LIDAR system 100described above or below may be incorporated into vehicle 110 to providea range-sensing vehicle. Specifically, in this example, LIDAR system 100integrates multiple scanning units 104 and potentially multipleprojecting units 102 in a single vehicle. In one embodiment, a vehiclemay take advantage of such a LIDAR system to improve power, range andaccuracy in the overlap zone and beyond it, as well as redundancy insensitive parts of the FOV (e.g. the forward movement direction of thevehicle). As shown in FIG. 6A, vehicle 110 may include a first processor118A for controlling the scanning of field of view 120A, a secondprocessor 118B for controlling the scanning of field of view 120B, and athird processor 118C for controlling synchronization of scanning the twofields of view. In one example, processor 118C may be the vehiclecontroller and may have a shared interface between first processor 118Aand second processor 118B. The shared interface may enable an exchangingof data at intermediate processing levels and a synchronization ofscanning of the combined field of view in order to form an overlap inthe temporal and/or spatial space. In one embodiment, the data exchangedusing the shared interface may be: (a) time of flight of receivedsignals associated with pixels in the overlapped field of view and/or inits vicinity; (b) laser steering position status; (c) detection statusof objects in the field of view.

FIG. 6B illustrates overlap region 600 between field of view 120A andfield of view 120B. In the depicted example, the overlap region isassociated with 24 portions 122 from field of view 120A and 24 portions122 from field of view 120B. Given that the overlap region is definedand known by processors 118A and 118B, each processor may be designed tolimit the amount of light emitted in overlap region 600 in order toconform with an eye safety limit that spans multiple source lights, orfor other reasons such as maintaining an optical budget. In addition,processors 118A and 118B may avoid interferences between the lightemitted by the two light sources by loose synchronization between thescanning unit 104A and scanning unit 104B, and/or by control of thelaser transmission timing, and/or the detection circuit enabling timing.

FIG. 6C illustrates how overlap region 600 between field of view 120Aand field of view 120B may be used to increase the detection distance ofvehicle 110. Consistent with the present disclosure, two or more lightsources 112 projecting their nominal light emission into the overlapzone may be leveraged to increase the effective detection range. Theterm “detection range” may include an approximate distance from vehicle110 at which LIDAR system 100 can clearly detect an object. In oneembodiment, the maximum detection range of LIDAR system 100 is about 300meters, about 400 meters, or about 500 meters. For example, for adetection range of 200 meters, LIDAR system 100 may detect an objectlocated 200 meters (or less) from vehicle 110 at more than 95%, morethan 99%, more than 99.5% of the times. Even when the object'sreflectivity may be less than 50% (e.g., less than 20%, less than 10%,or less than 5%). In addition, LIDAR system 100 may have less than 1%false alarm rate. In one embodiment, light from projected from two lightsources that are collocated in the temporal and spatial space can beutilized to improve SNR and therefore increase the range and/or qualityof service for an object located in the overlap region. Processor 118Cmay extract high-level information from the reflected light in field ofview 120A and 120B. The term “extracting information” may include anyprocess by which information associated with objects, individuals,locations, events, etc., is identified in the captured image data by anymeans known to those of ordinary skill in the art. In addition,processors 118A and 118B may share the high-level information, such asobjects (road delimiters, background, pedestrians, vehicles, etc.), andmotion vectors, to enable each processor to become alert to theperipheral regions about to become regions of interest. For example, amoving object in field of view 120A may be determined to soon beentering field of view 120B.

Example Implementation: Surveillance System

FIG. 6D illustrates the implementation of LIDAR system 100 in asurveillance system. As mentioned above, LIDAR system 100 may be fixedto a stationary object 650 that may include a motor or other mechanismsfor rotating the housing of the LIDAR system 100 to obtain a wider fieldof view. Alternatively, the surveillance system may include a pluralityof LIDAR units. In the example depicted in FIG. 6D, the surveillancesystem may use a single rotatable LIDAR system 100 to obtain 3D datarepresenting field of view 120 and to process the 3D data to detectpeople 652, vehicles 654, changes in the environment, or any other formof security-significant data.

Consistent with some embodiment of the present disclosure, the 3D datamay be analyzed to monitor retail business processes. In one embodiment,the 3D data may be used in retail business processes involving physicalsecurity (e.g., detection of: an intrusion within a retail facility, anact of vandalism within or around a retail facility, unauthorized accessto a secure area, and suspicious behavior around cars in a parking lot).In another embodiment, the 3D data may be used in public safety (e.g.,detection of: people slipping and falling on store property, a dangerousliquid spill or obstruction on a store floor, an assault or abduction ina store parking lot, an obstruction of a fire exit, and crowding in astore area or outside of the store). In another embodiment, the 3D datamay be used for business intelligence data gathering (e.g., tracking ofpeople through store areas to determine, for example, how many people gothrough, where they dwell, how long they dwell, how their shoppinghabits compare to their purchasing habits).

Consistent with other embodiments of the present disclosure, the 3D datamay be analyzed and used for traffic enforcement. Specifically, the 3Ddata may be used to identify vehicles traveling over the legal speedlimit or some other road legal requirement. In one example, LIDAR system100 may be used to detect vehicles that cross a stop line or designatedstopping place while a red traffic light is showing. In another example,LIDAR system 100 may be used to identify vehicles traveling in lanesreserved for public transportation. In yet another example, LIDAR system100 may be used to identify vehicles turning in intersections wherespecific turns are prohibited on red.

It should be noted that while examples of various disclosed embodimentshave been described above and below with respect to a control unit thatcontrols scanning of a deflector, the various features of the disclosedembodiments are not limited to such systems. Rather, the techniques forallocating light to various portions of a LIDAR FOV may be applicable totype of light-based sensing system (LIDAR or otherwise) in which theremay be a desire or need to direct different amounts of light todifferent portions of field of view. In some cases, such lightallocation techniques may positively impact detection capabilities, asdescribed herein, but other advantages may also result.

It should also be noted that various sections of the disclosure and theclaims may refer to various components or portions of components (e.g.,light sources, sensors, sensor pixels, field of view portions, field ofview pixels, etc.) using such terms as “first,” “second,” “third,” etc.These terms are used only to facilitate the description of the variousdisclosed embodiments and are not intended to be limiting or to indicateany necessary correlation with similarly named elements or components inother embodiments. For example, characteristics described as associatedwith a “first sensor” in one described embodiment in one section of thedisclosure may or may not be associated with a “first sensor” of adifferent embodiment described in a different section of the disclosure.

It is noted that LIDAR system 100, or any of its components, may be usedtogether with any of the particular embodiments and methods disclosedbelow. Nevertheless, the particular embodiments and methods disclosedbelow are not necessarily limited to LIDAR system 100 and may possiblybe implemented in or by other systems (such as but not limited to otherLIDAR systems, other electrooptical systems, other optical systems, etc.—whichever is applicable). Also, while system 100 is described relativeto an exemplary vehicle-based LIDAR platform, system 100, any of itscomponents, and any of the processes described herein may be applicableto LIDAR systems disposed on other platform types. Likewise, theembodiments and processes disclosed below may be implemented on or byLIDAR systems (or other systems such as other electrooptical systemsetc.) which are installed on systems disposed on platforms other thanvehicles, or even regardless of any specific platform.

Calibration for Reflected Light

In some configurations, one or more sensors configured to detect lightreflected from objects in a field of view (e.g., laser light, LED light,or any other type of light or EM radiation reflected from objects) mayexperience incident light from other sources. In some cases, incidentlight from these other sources may be undesirable and may cause noisethat may negatively impact the LIDAR system's ability to detect objects,obtain an intended resolution relative to detected objects, determineaccurate distance information relative to detected objects, etc. Suchsources of noise may include ambient light from an environment of theLIDAR system and/or light from one or more discrete sources in anenvironment of the LIDAR system. Other sources of noise that may impactoperation of the LIDAR sensors, and, therefore, the LIDAR system, mayinclude random noise originating from various components included in theLIDAR detection path (e.g., sensors, amplifiers, filters, etc.).

Still other sources of noise may include light that is projected by oneor more light sources of the LIDAR system, but which has not beenreceived by the LIDAR system after reflection from an object in theenvironment of the LIDAR system. Such light, which may result fromreflections internal to the LIDAR system, ambient lighting conditionswithin the LIDAR system during LIDAR light source activation, ambientlighting outside of the LIDAR system, coupling between lighttransmission (TX) and light reception (RX) paths (e.g., in a monostaticLIDAR configuration), or through other potential mechanisms other thanLIDAR light reflections from objects outside of the LIDAR system, may bereferred to as extraneous detected light. In other words, extraneousdetected light may include any light detected by one or more detectorsof the LIDAR system without that light having been reflected from anobject outside of the LIDAR system prior to its detection by the one ormore detectors of the LIDAR system. In many cases, a significantcomponent of the extraneous detected light may arise from internallyreflected light—i.e., light projected by the one or more light sourcesof the LIDAR system that does not escape from the LIDAR system, butrather reflects off of one or more surfaces internal to the LIDAR systembefore impinging upon a detecting sensor.

Extraneous detected light (which may include internally reflected light)may cause a variety of responses at a detector of the LIDAR system. Inmany cases, the type of response a LIDAR detector has to extraneousdetected light may depend on the type of light source or a manner ofoperating the light source of the LIDAR system. For example, detectorresponses to extraneous detected light in a continuous wave LIDAR systemmay be different than detector responses to extraneous detected light ina LIDAR system in which the LIDAR light sources are pulsed, for example.In some example cases, a response of a LIDAR detector to extraneousdetected light may appear as what may be referred to as a parasiticpulse. Examples of a parasitic pulse response of a LIDAR detector areshown by graph 700 and 710 in FIGS. 7A and 7B. The graphs in FIGS. 7Aand 7B are exemplary only. Many other waveform shapes or responses maybe generated by a LIDAR detector in response to any source of extraneousdetected light.

As noted above, a LIDAR light source (such as any of the light sourcesdescribed elsewhere in this disclosure) may be used to project lighttoward a field of view in order to illuminate one or more objects in thefield of view. The LIDAR system may include at least one sensorconfigured to detect light that is reflected from the one or moreilluminated objects in the field of view. Under an ideal scenario, thesensor may experience no other sources of incident light other than thelaser light reflected from one or more illuminated objects in the fieldof view. Under more typical situations, however, the sensor mayexperience light from several different sources. In some cases, lightincident on the LIDAR sensor may include LIDAR light reflected from thescene (e.g., LIDAR light source light reflected from a detected objectand incident upon the sensor at relevant timings to indicateTime-of-Flight detection); light originating from the scene (e.g.,ambient light fixed in time, light from time-varying sources, etc.);and/or light from internal reflections. Additionally, other sources ofnoise in the sensing system may exist, including random noise resultingfrom components of the detection path (e.g., sensors, amplifiers,filters, etc.).

Returning to FIG. 7A, graph 700 illustrates an example of a sensorresponse to internally reflected light incident upon the sensor. In somecases, such as a monostatic LIDAR implementation, this internallyreflected light may be sensed as a result of coupling between the lighttransmission and light reception paths. However, such a response may beobserved in any LIDAR system in which at least some portion of the lightfrom a LIDAR light source is incident upon a LIDAR sensor prior to thatlight being reflected from an object in the scene.

The output of a LIDAR detector in response to extraneous detected light,examples of which are provided by graph 700 of FIG. 7A and graph 710 ofFIG. 7B, may include certain characteristic elements. For example, insome cases, extraneous detected light (e.g., internally reflected light,etc.) may be detected and cause an initial rise in sensor signalamplitude (e.g., at peak 702). Such a detection peak (e.g., a parasiticpulse) can be problematic as it may saturate the sensor and render thesensor unable to detect or differentiate light reflected from an objectin the LIDAR FOV from light that is internally reflected within theLIDAR system.

In some cases, a sensor response to extraneous detected light may not belimited to a single output peak response. Instead, in some cases asshown by graph 710 in FIG. 7B, the sensor may exhibit a prolongedresponse to the extraneous detected light. Such a response may manifest,for example, as “ringing” at the sensor. This ringing may occur as aresult of the extraneous detected light exciting the sensor circuitry(e.g., including parasitic capacitances and inductances) such that thecircuitry resonates at a characteristic frequency. For example, as shownby graph 710, after an initial positive response to the incidentextraneous detected light, the sensor output may oscillate towardnegative output values resulting in negative peak 704. The LIDAR sensormay continue to ring for a period of time dependent upon dampingcharacteristics of the circuitry. For example, as shown in FIG. 7B,after peak 704, the sensor response may again oscillate toward positiveresponse values at peak 706 before eventually settling back to zeroamplitude.

Any LIDAR sensor response to extraneous detected light (two examples ofwhich are shown in FIGS. 7A and 7B) may adversely impact the detectioncapabilities of the LIDAR system. For example, any non-zero output ofthe LIDAR sensor originating from light sources other than lightreflected from objects in a LIDAR FOV may hinder the system's ability todifferentiate between components of the sensor output originating fromlight reflected from objects in the LIDAR FOV and those componentsoriginating from extraneous detected light (e.g., internally reflectedlight, etc.). With respect to the example shown in FIG. 7B, during theinitial sensor response to the extraneous detected light (e.g., peak702) and any subsequent effects (e.g., the ringing represented by peaks704 and 706, etc.), targets in a scene may be masked by the sensor'sresponse to the extraneous detected light. As a result, such targets maygo undetected or the detection quality may be compromised. Such maskingmay occur as a result of saturation of the sensor during the initialresponse to the extraneous detected light and during subsequent ringingof the sensor, or as a result of any other non-zero response of theLIDAR sensor(s) to extraneous detected light. Such masking may alsooccur at any time during the sensor response to the extraneous detectedlight (e.g., initial response or ringing) as a result of the extraneousdetected light response being greater than the sensor's response toincoming light reflected from the scene.

This masking may be especially problematic for detecting objects withincertain ranges of the LIDAR system that correspond with time periods ofinterest (e.g., those time periods corresponding to the time of flightfor light to and from an object in an operational range of the LIDARsystem (1 m to 500 m; 20 m to 250 m; etc.)). Such time periods mayoverlap with a LIDAR sensor's response to extraneous detected light(e.g., initial pulse 702 and/or ringing characteristics such as peaks704, 706) which may occur over time periods during which return lightreflections from objects in a LIDAR FOV are expected. For example, lighttravels about 0.3 meters per nanosecond. In some cases, a LIDAR sensormay exhibit at least some response to extraneous detected light over aperiod of 500 nanoseconds or more. Thus, in the example shown in FIG.7B, for example, and assuming that the sensor's response to extraneousdetected light continued for 500 nanoseconds, detection of objects up toabout 75 meters from the LIDAR system may be affected by the sensorresponse (e.g., 500 nanoseconds is the approximate time required for theprojected LIDAR light to travel to an object at a distance of 75 metersfrom the LIDAR system, reflect from the object, and return back to theLIDAR system). Objects closer in to the LIDAR system (e.g., a distanceroughly corresponding to a 250 nanosecond time of flight or less of anemitted LIDAR light pulse), may be partially or completely masked by thesensor response to extraneous detected light due to potentially largeramplitude responses of the sensor within the 250 nanosecond time range(e.g., peaks 702 as shown in FIGS. 7A and 7B). Of course, as noted, thesensor responses shown in FIGS. 7A and 7B are just two examples. In somecases, a particular sensor may exhibit a response (e.g., ringing) for alonger or shorter amount of time in response to extraneous detectedlight or may exhibit lower or higher amplitude responses to incidentextraneous detected light.

In some embodiments, detection of close range targets (or any othersaffected by a sensor response to extraneous detected light, light noise,or noise from other sources) may be improved by accounting for sensorresponses other than those resulting from LIDAR light reflected fromtarget objects. In some cases, the effects of the sensor responses tounwanted light input or noise can be reduced (e.g., subtracted, canceledout, or otherwise accounted for) such that the sensor response toincident LIDAR light reflected from objects in a scene may be morediscernable. Such an approach may aid in the detection of objects andthe determination of ranging information relative to those objects evenwhere those objects are associated with LIDAR light flight times thatcoincide with times during which noise is present on the output(s) ofLIDAR sensor(s).

In order to identify sensor responses not attributable to LIDAR lightreflected from objects in a scene, any suitable approach may be used.For example, in some embodiments, another sensor (e.g., a calibrationsensor) may be included in the LIDAR system. This calibration sensor maybe used to sense the extraneous detected light from a LIDAR light sourceor any any other source of noise or unwanted sensor response in theLIDAR sensing system. In some cases, the calibration sensor may beshielded (e.g., partially or fully) from LIDAR light that returns to theLIDAR system after reflection from one or more objects in the LIDAR FOV.Such shielding may be provided by one or more structures arranged toblock the calibration sensor from returning light or may be provided byplacing the calibration sensor outside of the optical path of thereflected LIDAR light returning from the LIDAR FOV.

Thus, the output of the calibration sensor may exclude componentsassociated with LIDAR light reflected from objects in a scene, but mayinclude responses to extraneous detected light and/or any other sourcesof noise in the LIDAR sensing system. As a result, the output of thecalibration sensor may be used as a guide for filtering the output ofone or more imaging sensors. In some cases, for example, the output ofthe calibration sensor may indicate a sensor response to directlydetected light, and this response may be subtracted from an observedresponse of one or more of the LIDAR imaging sensors that includesresponses to received LIDAR light reflected from the FOV. This type ofsubtraction may provide conditioned outputs from one or more imagingsensors that is free or nearly free of unwanted effects of extraneousdetected light. Effects of ambient light or other sources of noise maybe treated similarly by observing the effects with a calibration sensorand subtracting those effects from the output of an imaging sensor. Inthis way, detection and ranging relative to target objects may beimproved, especially for objects at relatively close range to a LIDARsystem (e.g., within 10 meters, 20 meters, 50 meters, 100 meters, etc.).

FIG. 8A provides a diagrammatic representation of a LIDAR system 802,according to disclosed embodiments. For example, system 802 may include,among other elements, a processor 804, a light sensor 806, asupplemental light sensor 808 (e.g., a calibration sensor in someembodiments), a light guide 810, a light source 812, and a lightdeflector 814. In some cases, processor 804 may have access to a memory816, which may include one or more databases, instructions forexecution, etc.

In some cases, processor 804, which may include one or more processingunits similar to processing unit 108 described above, may control theoperation of light source 812 to selectively emit a light 813 (e.g., aLIDAR laser emitted continuously, pulsed, etc.) for illuminating atleast a portion of a LIDAR FOV 820. As described in detail above, alight deflector 814 (which may be similar to light deflector 114 ofscanning unit 104) may be controlled by processor 804 to move (e.g., bycontrolled rotation along two or more axes) in order to scan light beam813 projected from light source 812 over LIDAR FOV 820. At eachinstantaneous position of the deflector 814, light reflected from one ormore objects located in a corresponding region of the LIDAR FOV 820 maybe incident upon deflector 814 and may be passed along toward thesensing elements of the LIDAR system.

In some cases, the LIDAR sensing system may include sensor 806, whichmay be similar to sensor 116 described above. Light guide 810 mayinclude one or more components configured to pass light or toselectively alter the path of light. In some cases, light guide 810 mayinclude one or more reflective elements (e.g., mirrors, mirrored faces,etc.), lenses, beam splitters, etc. for guiding light 813 to deflector814 and for guiding laser light reflections received from the LIDAR FOVby deflector 814 toward sensor 806. Similar to the techniques describedabove, sensor 806 may generate LIDAR reflection signals corresponding tolevels of reflected light incident upon sensor 806, and these signalsmay be collected and used by processor 804 to generate LIDAR images(e.g., point clouds where each point represents data collected for aparticular FOV pixel) and to obtain ranging information relative toobjects occurring in an environment of the LIDAR system within thescanned FOV 820 (e.g., each point in the point cloud may includedistance information to an object or portion of an object occurring at aparticular FOV pixel).

As noted, some embodiments may include supplemental sensor 808, whichmay be similar to or the same as sensor 806. Supplemental sensor 808 maybe used to generate an output to extraneous detected light (e.g.,ambient light or internally reflected light within the LIDAR systemoriginating from operation of the light source 812). For example, asshown in FIG. 8A, sensor 808 may receive extraneous detected light inthe form of internally reflected light rays 850, which may result fromat least some portion of beam 813 (e.g., light ray 852) being incidentupon an internal surface of a LIDAR system housing, wall, etc. Whilesensor 808 receives internally reflected light from the light source812, it does not receive any reflected light from the LIDAR FOV (i.e.,light 822). As a result, the output from sensor 808 is indicative of thelevel of internally reflected light within the LIDAR system without anycontribution from reflected LIDAR light received from the LIDAR FOV. Theoutput of sensor 808 may be used, for example, by processor 804 toremove (e.g., subtract) a similar response of sensor 806 to extraneousdetected light (e.g., including internally reflected light representedby light ray 854) from source 812.

FIG. 8E diagrammatically illustrates light components incident uponsensors 806 and 808. For example, light reflected from the LIDAR FOV(e.g., light 822) is incident upon sensor 806 of FIG. 8A, but is notincident on calibration sensor 808. Extraneous detected light (e.g.,internally reflected light) 849, however, includes components receivedby both sensor 806 and 808. For example, internally reflected lightcomponents 850 are received by sensor 808, and internally reflectedlight components 854 are received by sensor 806.

Thus, in operation, the at least one processor 804 may be configured tocontrol at least one light source (e.g., source 812) for projectinglight toward a field of view 820. As shown in FIG. 8A, processor 804 mayreceive from at least one first sensor (e.g., sensor 806) first signalsassociated with light projected by the at least one light source. Thesefirst signals from sensor 806 may include components resulting fromreceived light that is reflected from an object in the field of view(e.g., light 822 reflected from objects in the FOV 820). These firstsignals may also include components, for example, associated withextraneous detected light (e.g., internally reflected light 854, orother light effects not associated with LIDAR light reflections from anobject in the LIDAR FOV).

Processor 804 may also be configured, as shown in FIG. 8A, to receivefrom a sensor, such as calibration sensor 808, second signals associatedwith light projected by the at least one light source. In contrast tothe first signals received by processor 804 from sensor 806, thesesecond signals received from sensor 808 do not include any componentsresulting from received light that is reflected from an object in thefield of view (e.g., light 822 reflected from objects in the FOV 820).Rather, these second signals include, at least as a primary component,signals resulting from extraneous detected light (e.g., internallyreflected light 850, or other light effects not associated with LIDARlight reflections from an object in the LIDAR FOV). To correct for theeffects of internally reflected light or other instances of extraneousdetected light, processor 804 may correct the output from sensor 806based on the output of sensor 808. In some cases, the correction mayinvolve substracting the output of sensor 808 (e.g, with appliedweighting factors, after signal conditioning, etc.) from the output ofsensor 806. In this way, the unwanted effects of extraneous detectedlight may be reduced or eliminated from the output of sensor 806.

As noted above, the at least one processor 804 may be configured tocoordinate at least one light deflector 814 and the at least one lightsource 812 such that when the at least one light deflector 814 assumes aparticular instantaneous position, a portion of a light beam 813 isdeflected by the at least one light deflector 814 from the at least onelight source towards an object in the field of view 820, and reflectionsof the portion of the light beam from the object are deflected by the atleast one light deflector toward at least one sensor (e.g., as light822).

Various other configurations of the LIDAR system may be employed. Forexample, as shown in FIG. 8B, a LIDAR system 832 may include componentssimilar to LIDAR system 802. In some cases, LIDAR system 832 mayinclude, among other elements, processor 804, light sensors 806A and806B, supplemental light sensor 808 (e.g., a calibration sensor in someembodiments), light guide 810, a light deflector 814, and a memory 816.In contrast to LIDAR system 802, which as illustrated includes only asingle light source 812, LIDAR system 832 may include multiple lightsources. For example, system 832 may include a first light source 812Aand a second light source 812B. Any number of additional light sourcesmay also be employed.

In some embodiments, both light source 812A and light source 812B may beaimed at deflector 814 or may otherwise provide light output directly orindirectly to light deflector 814. In such embodiments, light source812A may be associated with a light output 813A, and light source 812Bmay be associated with a light output 813B. Processor 804 may beconfigured to control deflector 814 such that when deflector 814 assumesa particular instantaneous position, light 813A and light 813B from theplurality of light sources 812A and 812B is projected towards aplurality of independent regions in the field of view. For example,light output 813A may be projected toward region 824, and light output813B may be projected toward region 825.

As noted, processor 804 may develop a LIDAR image (e.g., point cloud)and/or ranging information relative to objects in the LIDAR FOV based onacquired reflections of LIDAR light that are received after reflectingfrom objects in the LIDAR FOV. These light reflections may be acquired,for example, using deflector 814, light guide 810, and/or sensors 806Aand 806B. In addition to light reflections received from the LIDAR FOV,sensors 806A and 806B may also receive extraneous detected light (e.g.,in the form of internally reflected light 854 and 855, respectively,produced by internal reflections of light rays 852A and 852B).Calibration sensor 808, on the other hand, may be shielded from lightreflections received from the LIDAR FOV and, instead, may only receiveextraneous detected light (e.g., in the form of internally reflectedlight 850 produced by internal reflections of light rays 852A and 852B).As described relative to the embodiment of FIG. 8A, the output ofcalibration sensor 808 may be used to correct or otherwise condition theoutput of sensors 806A and/or 806B to reduce or eliminate the effects ofextraneous detected light.

While the light reflections incident upon a described light sensor(e.g., any of sensors 806, 806A, 806B, 808, etc.) may result in variouspatterns, in some cases, the incident light reflections may form a lightspot having an outer boundary 877. Such a light spot 860, for example,is shown in FIG. 8C. While light spot 860 is shown in FIG. 8C as havinga generally circular shape, light spot 860 may have various shapes,including elliptical, irregular, etc. In some cases, a light intensitylevel may be substantially uniform over an area of sensor 806 affectedby light spot 860. In other cases, however, the intensity of incidentreflected light constituting light spot 860 may vary over the areaassociated with light spot 860. For example, light spot 860 may havecertain areas that receive higher light intensities than other areas.While higher light intensity areas may occur at any location withinlight spot 860, in the example show in FIG. 8C, an area 862 near acenter region of light spot 860 may be associated with a certain lightintensity, which may be greater than a light intensity associated withadjacent area 864. Similarly, area 864 may exhibit a higher lightintensity level than an intensity level associated with area 866, andarea 866, in turn, may be associated with a light intensity level higherthan a light intensity level associated with area 868.

In some cases, light intensity levels may vary smoothly over thetwo-dimensional plane represented by sensor 806. As a result, theboundary lines separating areas 862, 864, 866, and 868 from one anothermay designate, for example, isolines consisting of points having thesame light intensity level (e.g., similar to isobars on a pressure map)at a given time or on average over a given period. Thus, each of contourlines 872, 874, 876, and 878, as shown in FIG. 8C, may represent a linethrough points of equal light intensity.

FIG. 8D provides a representation of an example light intensitydistribution over sensor 806 across the line A-A shown in FIG. 8C. Asrepresented by the example illustrated in FIG. 8D, the light intensitydistribution resulting from an incident light beam received subsequentto reflection from an object in a LIDAR FOV, in some cases, mayapproximate a Gaussian distribution, although any other type ofdistribution may also be encountered. FIG. 8D shows the iso-intensitylevels associated with each of boundary lines 872, 874, 876, and 878.FIG. 8D also provides an exemplary intensity distribution profile thatmay be encountered in each of areas 862, 864, 866, and 868 of sensor806. Line 880 may represent effects of light incident on a sensor otherthan light reflected from objects in a LIDAR FOV. For example, in somecases, line 880 may conceptually represent the effects of extraneousdetected light (e.g., internally reflected light, ambient light, ornoise from other sources).

As noted above, light spot 860 may include an outer boundary 877. Insome cases, outer boundary 877 may represent a boundary beyond whichsubstantially no reflected light constituting light spot 860 is receivedat sensor 806. That is, in some instances, light sensitive elementsassociated with sensor 806 that fall outside of outer boundary 877 mayreceive no LIDAR light reflected from objects in the LIDAR FOV. In othercases, however, outer boundary 877 may correspond to a lightiso-intensity level (or threshold) that is significantly reduced, forexample, with respect to a peak light intensity level associated withlight spot 860. For example, as shown in FIGS. 8C and 8D, outer boundary877 may correspond with boundary line 878. Beyond boundary line 878,some reflected light is received at sensor 806, but in this area, thereflected light received at sensor 806 has an intensity levelsignificantly reduced relative to a peak intensity value associated witharea 862, for example. Outer boundary 877, when not corresponding to anabsolute light intensity boundary beyond which no reflected light isincident upon sensor 806, may be associated with various light intensitylevels that may be expressed as a percentage of a maximum lightintensity level associated with light spot 860. For example, in somecases, outer boundary 877/boundary 878 may correspond to a lightintensity level that is less than 1%, 2.5%, or 5% of a peak lightintensity level associated with light spot 860. In other cases, outerboundary 877/boundary 878 may have a light intensity level that is up to10% of a peak light intensity level (or even up to 15%, 20% or 25% of apeak light intensity level) associated with light spot 860.

Returning to FIGS. 8A and 8B, in some embodiments, calibration sensor808 may be provided as a sensor unit separate from and spaced apart fromsensors 806, 806A, or 806B. In such cases, the reflected LIDAR lightreceived back from a scene may be made incident upon sensor 806 (e.g.,as shown in FIG. 8C), such that sensor 806 may produce output signalsindicative of the received reflected LIDAR light. On the other hand,calibration sensor 808 may be isolated from the reflected LIDAR lightsuch that substantially none of the received LIDAR light reflections areprovided to calibration sensor 808. Instead, calibration sensor 808 mayreceive light noise (e.g., internally reflected light, etc.) and maygenerate signals responsive to the received light noise.

Such light noise may originate from multiple different sources. In somecases, as described above, the light noise received at calibrationsensor 808 may include extraneous detected light, which may result atcalibration sensor 808 through similar mechanisms that cause theextraneous detected light to be received at sensor 806 (e.g., internalreflection, coupling between transmission and reception paths, couplingbetween optical elements in the light project path/light guiding path,etc.). The light noise received at calibration sensor 808 may alsoinclude light received from ambient light within the LIDAR system, inthe environment of the LIDAR system, or any other source of light thatmay produce or contribute to unwanted light noise at the detectors ofthe LIDAR system.

While the calibration sensor may be separate from sensor 806, in somecases, the sensor or sensor element used for calibration may be includedwith or may form part of sensor 806. For example, in some cases acalibration sensor 887 (FIG. 8C) may constitute a single sensor element(e.g., a single SPiM comprising a group of light sensitive elements suchas avalanche photo diodes, etc.) of sensor 806. In other cases, acalibration sensor 888 may include a plurality of sensor elements ofsensor 806. The plurality of sensor elements selected for inclusion incalibration sensor 888 may include a contiguous group of sensor elements(as shown in FIG. 8C) or may include two or more sensor elementsassociated with sensor 806 that are spaced apart from one another.Notably, in embodiments where a calibration sensor constitutes one ormore sensor elements of sensor 806, those sensor element(s) selected foruse in calibration may fall outside of outer boundary 877 and/orboundary line 878.

Regardless of whether the calibration sensor constitutes a part ofsensor 806 or whether it is separate from sensor 806, the output of thecalibration sensor may be used to mitigate unwanted effects of lightnoise on the LIDAR system. For example, processor 804 may receivesignals generated by calibration sensor 808, 887, 888, etc. and based onthose received signals, processor 804 may determine an indicator of amagnitude of light noise at the calibration sensor. Such an indicatormay have various forms. In some embodiments, the indicator may include aparameter value associated with any of the signals generated by thecalibration sensor (e.g., amplitude, current level, voltage level,etc.). In other cases, the indicator may be any indirectly determinedquantity or value that may be associated with an intensity levelincident upon the calibration sensor, etc.

With the determined indicator of the magnitude of light noise at thecalibration sensor, processor 804 may condition one or more signalsoutput from sensor 806 (or 806A, 806B) in response to incident lightspot 860. In other cases, the signals output from sensor 806 (or 806A,806B) may be used together with the determined indicator to arrive at anoutput from the LIDAR system or a subsystem of the LIDAR system that isat least partially corrected for light noise effects. For example, basedon the determined indicator and the signals received from sensor 806, adistance may be determined to an object in the LIDAR FOV from whichreflected LIDAR light was received. The distance calculation thataccounts for light noise may be more accurate. In some cases, distancecalculations that were otherwise not possible in view of light noise maybecome possible through correction based on the outputs from sensor 806(or 806A, 806B) and calibration sensor 808, 887, 888. The same principlemay also hold true for object detection. That is, in some cases, thepresence of light noise may mask an object from detection by the LIDARsystem or may make detection difficult or unreliable. Removing at leastsome of the effects of light noise may result in more object detectionsand higher confidence in object detections. Conditioning the outputsignals from sensor 806 (or otherwise accounting for light noise) basedon the indicator of light noise determined based on the output ofcalibration sensor 808, 887, 888 may at least partially (or fully)remove the effects of light noise from the signals output from sensor806.

As noted, in some embodiments, the mitigation of light noise effects maybe accomplished by processor 804 being configured to correct the signalsreceived from sensor 806 (or 806A, 806B) using the indicator of lightnoise determined based on the output of the calibration sensor 808, 887,888. Correction of the signals from sensor 806 (or 806A, 806B) mayinclude any appropriate correction technique. In some cases, processor804 may determine a sensor response to light noise from calibrationsensor 808, 887, 888. With an assumption that the calibration sensor808, 887, 888 has a response to the light noise similar to or the sameas sensor 806 (or 806A, 806B), processor 804 may subtract the output ofcalibration sensor 808, 887, 888 (or an approximation of that output, afunction or curve generated based on the output, etc.) from the outputof sensor 806 (or 806A, 806B).

Such as correction may be performed at any point in time duringoperation of the LIDAR system. In some cases, the correction may beperformed periodically during operation of the LIDAR system. Forexample, as the deflector is scanned through each of the instantaneouspositions associated with different locations of the LIDAR FOV, acorrection of the output of sensor 806 (or 806A, 806B) may be determinedrelative to each of the instantaneous positions based on the output ofthe calibration sensor 808, 887, and/or 888. In practice, processor 804may determine based on the output of calibration sensor 808, 887, and/or888 a plurality of indicators, where each indicator represents amagnitude of light noise associated with a corresponding, particularinstantaneous position through which the deflector is scanned. Processor804 may then use the determined plurality of indicators to correct thesignals received from sensor 806 (or 806A, 806B). Additionally, for eachof the instantaneous positions through which the deflector is scanned,processor 804 may store the determined indicator value or valuesindicative of an amount of light noise associated with the instantaneousposition of the deflector/particular region of the LIDAR FOV.

In addition to correcting the output of sensor 806 (or 806A, 806B) basedon the calibration sensor output (or otherwise determining an actualcontribution of LIDAR light reflections to the sensor 806 output (freeor partially free of light noise) based on the output of the calibrationsensor(s)), other uses of the calibration sensor output may also berealized. For example, in some cases, the output of the calibrationsensor(s) may be used to monitor performance characteristics of LIDARsystem components. In some cases, processor 804 may monitor anddetermine performance changes of at least one sensor (e.g., sensor 806)over a period of time. By storing light noise information (e.g., adetected magnitude associated with a extraneous detected lightassociated with a LIDAR light source) at various time periods (e.g.,daily, weekly, monthly, etc.), information may be inferred regarding theoperation of the light source. For example, if there are changes overtime in the amplitude or other characteristics of the extraneousdetected light, processor 804, e.g., may infer that the LIDAR lightsource is deteriorating or operating outside of specified or intendedparameters. Additionally, especially in cases where the calibrationsensor constitutes a part of sensor 806 (e.g., sensor element 887 orelements 888), a change over time in measured magnitude or othercharacteristic of the extraneous detected light may indicatedeterioration of sensor 806 or operation of sensor 806 outside normalparameters.

Various configurations of calibration sensors may be used in thepresently disclosed LIDAR systems. As noted above, the calibrationsensor may be included as part of a LIDAR sensor (such as sensor 806).In other cases, the calibration sensor may be separate from the LIDARsensor, such as sensor 806, configured to receive reflections of LIDARlight from objects in a field of view. In such cases, the calibrationsensor may be similar to the sensor that receives the LIDAR lightreflections. Such similarity, for example, may facilitate correlationbetween the output of the LIDAR reflections sensor (e.g., sensor 806)and the calibration sensor and may facilitate correction of the outputof the LIDAR reflections sensor. In some instances, the calibrationsensor may be of a same type as the main LIDAR sensors (e.g. SiPM, APD),but this is not necessarily so, as the calibration sensors may also beformed of sensor types or configuration arrangements different from themain LIDAR sensors. Further, in some embodiments, the calibration sensormay be caused to operate with the same optical/electric/operationalcharacteristics as the main LIDAR sensors, but again, this is notnecessarily so. Nonetheless, in one embodiment a calibration sensor maybe employed that has an identical or nearly identical configuration asthe main LIDAR sensor (e.g., formed of the same type and number of lightsensitive elements, etc.) and may be operated according to identical ornearly identical parameter values as used to operate the main LIDARsensor. In this way, the calibration sensor(s) may exhibit the same orsimilar behavior, light input response, dynamic range, blinding, etc. asthe main LIDAR sensor(s), which may facilitate correction for lightnoise according to the processes described above.

In some embodiments, one or more calibration sensors included in theLIDAR system may have a configuration different from a main LIDAR sensoror may be operated differently, as compared to a main LIDAR sensor. Forexample, in some cases, a main LIDAR sensor (e.g., sensor 806) mayinclude a sensor array including more than one type of sensor. Suchsensor arrays, for example, may include SiPM sensor elements for longerrange detection, APD sensor elements for nearer field detection, etc. Insome cases, a sensor array may include one or more sensor elementsoperated using operational setting different from those employedrelative to one or more other sensor elements. Multiple calibrationsensors may be used. For example, a different calibration sensor may beincluded for each different type of main LIDAR sensor or set ofdifferent operational settings of a main LIDAR sensor. In other cases,however, a LIDAR system may include fewer calibration sensors than mainLIDAR sensors. For example, in some embodiments a LIDAR system mayinclude a single calibration sensor along with two or more differenttypes of main LIDAR sensors/sensor elements or may include a singlecalibration sensor where sensor elements of the LIDAR system areoperated according to different parameter settings.

Various techniques may be used for correlating the output of a singlecalibration sensor with multiple different main LIDAR sensor types ormultiple different operational settings for different main LIDARsensors. For example, based on predetermined testing or on-the-flycalibration procedures, the response of a single calibration sensor maybe correlated with multiple different main LIDAR sensor types ormultiple different operational settings for different main LIDARsensors. Such correlation, for example, may be obtained by observing andstoring (e.g., in one or more lookup tables) a response of thecalibration sensor to a certain lighting condition and by observing andstoring responses of the main LIDAR sensors to the same lightingconditions, relative positions with respect to light sources, etc. Then,during LIDAR system operation, processor 804 may correct the output ofone or more main LIDAR sensors based on an observed output of thecalibration sensor by accessing populated lookup tables anddetermining/accessing a stored correlation between the calibrationsensor and the type of main LIDAR sensor used and/or the set ofoperational parameters used for each of the main LIDAR sensors. In thisway, processor 804 may be configured to apply differing corrections todiffering ones of a plurality of detection elements (e.g., included insensor 806), relative to different sensor units, sensor unit detectionelements, or relative to different detection elements or sensor unitsbeing operated according to different operational parameter values. Incases where a main LIDAR sensor includes a plurality of detectionelements, processor 804 may associate a different correction factor(e.g., based on observed outputs from one or more calibration sensors)to each of the the plurality of detection elements within the main LIDARsensor.

As noted above, the calibration sensor may be part of or formedseparately from a main LIDAR sensor (e.g., sensor 806) and may includesimilar detection elements to those included in the main LIDAR sensor.In some embodiments, for example, at least one sensor 806 may include aplurality of detection elements of a single type (e.g., SiPM, APD, SPAD,PIN diode, etc.), and at least one calibration sensor may include atleast one detection element of the same type. In some cases, at leastone sensor 806 may include a plurality of detection elements includingmore than one type of detection element (e.g., detection elements of afirst type, such as APDs, along with detection elements of a secondtype, such as PIN diodes, etc.). In such cases, the at least onecalibration sensor may include at least one detection element of thefirst type and at least one detection element of the second type.

Moreover, as noted above, a LIDAR system may include more than onecalibration sensor. For example, a LIDAR system may include a pluralityof calibration sensors 808, 887, and/or 888. Each of the includedcalibration sensors may be located outside an outer boundary 877 oflight spot 860. In some cases, each calibration sensor may be be pairedwith one or more different LIDAR sensors 806 (or 806A, 806B). On theother hand, in some cases, more than one calibration sensor may bepaired with a single LIDAR sensor 806 (or 806A, 806B). For example,processor 804 may correct (or otherwise account for light noiseassociated with) the output of a single LIDAR sensor using a combinationof two or more different calibration sensors 808, 887, and/or 888, etc.In some cases, different weighting factors may be associated with eachavailable calibration sensor such that a correction of a particularsensor 806 output may depend more upon one calibration sensor than uponanother calibration sensor. In this way, a plurality of calibrationsensors (e.g., a combination or number of sensors 808, 887, and/or 888,etc.) may be used by processor 804 to determine a plurality ofcorresponding indicators of light noise. Processor 804 may then usethose plurality of indicators to correct the signals received fromsensor 806, for example, to subtract or account for the effects of lightnoise on sensor 806.

As noted, certain LIDAR systems may employ at least one calibrationsensor and at least one main sensor. The calibration sensor and the mainsensor may both be formed of the same type or configuration of detectoror they may be formed of different types of sensor or configureddifferently. The calibration sensor and the main sensor may be formed onthe same die or substrate or may be formed on different die orsubstrates. Regardless of the configuration of calibration sensor(s) andmain LIDAR sensor(s), the LIDAR system may be capable of accounting fordifferences in responses between any two or more sensors. Such acapability may facilitate compensation for light noise in one or moresensors based on the output observed in one or more different lightsources, as even identical or nearly identical sensors can havedifferent responses to sources of light noise (e.g., where sensors arepositioned in different locations or orientations, etc.).

In some embodiments, a translation function may be determined for tow ormore sensors that may enable translation between a signal of acalibration sensor, for example, to a signal of a relevant “main” pixelof the main LIDAR sensor. For example, a suitable translation functionmay have the form: Y1(t)=g·Y2(t−τ), where Y1, Y2 represent the relevantsignals, and g and ti represent translation coefficients to bedetermined. t is the sampling time (e.g. stored as a vector element). Itshould be noted that g and τ can change with time (i.e. drift) and withlaser power. Such changes may occur non-linearly. The translationcoefficients can be computed at any suitable times. In some cases, thetranslation coefficients may be determined for pulses/frames in which notargets are detected (e.g., in the near-field). Then, the translationcoefficients may be used or applied in response to LIDAR pulses andsensor outputs to account for light noise effects during times in whichtargets exist in a scanned portion of the LIDAR FOV (which would renderthe translation difficult or impossible to determine directly). Incertain cases, the system may include an optical barrier for preventingsome light reaching this sensor, but not necessarily so. Such an opticalbarrier may aid in determining relevant translation functions betweensensors.

The present disclosed embodiments also may include a method for using aLIDAR system to detect objects. Such a method may employ any of thetechniques described above for accounting for effects of light noise ona LIDAR sensor. For example, as outlined in FIG. 9, an exemplary methodmay include a step 901 of controlling at least one light source forprojecting light toward a field of view. At step 903, processor 804 mayreceive from at least one first sensor (e.g., sensor 806) signalsassociated with light projected by the at least one light source andreflected from an object in the field of view. Reflected light impingingon the at least one first sensor may be in a form of a light spot havingan outer boundary. At step 905, processor 804 may receive from at leastone second sensor (e.g., calibration sensor 808, 887, and/or 888),signals associated with light noise, wherein the at least one secondsensor is located outside the outer boundary of the light spot. At step907, processor 804 may determine, based on the signals received from theat least one second sensor, an indicator of a magnitude of the lightnoise. At step 909, processor 804 may determine, based on the indicatorand the signals received from the at least one first sensor, a distanceto an object in the LIDAR FOV. The presently disclosed embodiments mayalso include a non-transitory computer-readable storage medium includingstored instructions that, when executed by at least one processor, causethe at least one processor to perform a method for compensating forlight noise in a LIDAR system. The method may include any of the stepsshown in the flow chart of FIG. 9.

Pattern Recognition Used to Characterize LIDAR Window Obstruction

In a bi-static system, e.g., a LIDAR system and/or RADAR system,consistent with embodiments of the present disclosure, obstructions onthe protective window of the system may block light passage through theprotective window. For example, the vehicle may come in contact withsalt, mud, road grime, snow, rain, dust, bug debris, pollen, and birddroppings (among other things) which may block light from passingthrough the protective window of the system. Such blockages of light maybe complete or partial. For example, in some cases, the blockage may besubstantially opaque or, alternatively, may be translucent orsemi-transparent and may allow at least some light to pass. In somecases, the blockage may limit an amount of incident light throughrefraction of light (e.g., especially away from an intended lightreception path or away from relevant sensors). In such cases, theblockage may even be transparent. Additionally, a blockage may occuronly over a portion of the protective window relevant to the system ormay be more widespread. In some embodiments, discussed in detail below,a type of obstruction may be determined, and one or more remedialactions may be taken. For example, in some cases, an obstruction patternmay be detected by the system, and based on this pattern, the system mayclassify the obstruction and implement a process for cleaning theobstruction based on the classification. For example, based on thedetection and/or classification of the obstruction pattern, the systemmay modify an illumination scheme, a scanning scheme, a detection schemeor any other operational parameters of the system based on the resultsof the analysis of the obstruction.

FIG. 10A illustrates an exemplary system 1000 for detecting blockagesand/or impaired sensing including a projecting unit 1028, a scanningunit 1004, a sensing unit 1006, a processing unit 1008, and a protectivewindow 1010. System 1000 may be included in a LIDAR system, RADARsystem, other bistatic system, or the like. Consistent with embodimentsof the present disclosure, projecting unit 1028 may include at least onelight source 1012, scanning unit 1004 may include at least one lightdeflector 1014, sensing unit 1006 may include at least one sensor 1016,and processing unit 1008 may include at least one processor 1018. In oneembodiment, at least one processor 1018 may be configured to coordinateoperation of the at least one light source 1012 with the movement of atleast one light deflector 1014 in order to scan a field of view 1020.During a scanning cycle, each instantaneous position of at least onelight deflector 1014 may be associated with a particular portion 1022 offield of view 1020. In addition, system 1000 may include at least oneoptional optical window 1024 for directing light projected towards fieldof view 1020 and/or receiving light reflected from objects in field ofview 1020. Optional optical window 1024 may serve different purposes,such as collimation of the projected light and focusing of the reflectedlight. In one embodiment, optional optical window 1024 may be anopening, a flat window, a lens, or any other type of optical window.

In some embodiments, the above described elements of system 1000 may besimilar to or the same as those described with reference to FIG. 1A. Forexample, projecting unit 1028, scanning unit 1004, sensing unit 1006,and processing unit 1008 may be the same as projecting unit 102,scanning unit 104, sensing unit 106, and processing unit 108,respectively.

System 1000, may include a processor 1018, which may be processor 118 asdescribed with reference to FIGS. 5A-5C. In some embodiments, the system1000 may be a LIDAR system 100 mounted on a vehicle 110, as shown withreference to FIG. 1A. In some embodiments, system 100 may also be, orinclude, a processor of a host (i.e., higher level system). For example,system 1000 may be included in a LIDAR system 100. Processor 1018 may beseparate from processor 118, which executes the distance measurement orother LIDAR-specific actions. If processor 1018 is separate fromprocessor 118, processor 1018 may not control the light source of LIDARsystem 100 and may be configured to communicate with processor 118 oranother component of LIDAR system 100.

System 1000 may include a protective window 1010 disposed between atleast one component of the system 1000 (e.g., the sensor 1016) and ascene to be imaged. The protective window 1010 may include any lighttransmissive medium through which light (e.g., laser light transmittedto a scene to be imaged, reflected laser light received from the scene,ambient light, light from an internal light source, etc.) may at leastpartially pass. In some embodiments, the protective window 1010 isincluded as a component of system 1000. Additionally or alternatively,the protective window 1010 may be associated with a platform upon whichsystem 1000 is associated (e.g., a vehicle). In still other embodiments,the protective window 1010 may include light transmissive componentsfrom both system 1000 and the platform upon which system 1000 isdeployed. In some embodiments, the protective window 1010 may includeoptical window 1024 of the system 1000. In some embodiments, e.g., inwhich the system 1000 is mounted in the interior of a vehicle, theprotective window 1010 may be or include the windshield or a window ofthe vehicle. The protective window 1010 may be formed of any suitablematerial for passing at least some light therethrough. In someembodiments, the protective window 1010 or any of its components may bemade from glass, plastic, or any other suitable material, such asmaterials having low refractive indices. The protective window 1010 maybe flat, curved, or of any other shape and may possibly serve an opticalpurpose in addition to being protective (e.g. collimating light,filtering certain wavelengths, and so on), but not necessarily so.

System 1000 may include one of more light deflectors, e.g., deflectors1014, which may be deflectors 114A, 114B, and/or 214. The processor 1018may be configured to control the at least one light deflector 1014 suchthat during a scanning cycle of the field of view 1020, the at least onelight deflector 1014 assumes a plurality of instantaneous positions.Additionally, the processor 1018 may be configured to coordinateoperation of the at least one light deflector 1014 and the at least onelight source 1012 such that when the at least one light deflector 1014assumes a particular instantaneous position, a portion of a light beamis deflected by the at least one light deflector 1014 from the at leastone light source 1012 towards an object in the field of view 1020, andreflections of the portion of the light beam from the object aredeflected by the at least one light deflector 1014 toward at least onesensor 1016. The system 1000 may include a plurality of light sourcesaimed at the at least one light deflector 1014, wherein the at least oneprocessor 1018 is further configured to control the at least one lightdeflector 1014 such that when the at least one light deflector 1014assumes a particular instantaneous position, light from the plurality oflight sources 1012 is projected towards a plurality of independentregions, e.g., regions 1022, in the field of view 1020.

In some embodiments, the processor 1018 may be configured to control atleast one light source 1012 of the system 1000. As previously described,the processor 1018 may receive reflection signals from at least onesensor, e.g., sensor 1016. Reflection signals may include indications oflight incident on the at least one sensor 1016. In some cases, theincident light may include light reflected from the protective window1010. Additionally or alternatively, the incident light may includelight reflected from objects in the field of view 1020. This reflectedlight may pass through the protective window 1010 prior to reaching theat least one sensor 1016.

The presently disclosed system may operate in conjunction with lightsupplied by a primary light source, e.g., light source 1012 of thesystem 1000. That is, light incident on the sensor and used to determinethe presence of a blockage on a protective window, at type of blockage,etc., may include light that originated from the primary light source1012 of system 1000. For example, in some embodiments, a bistatic LIDARsystem (e.g., a system including different transmission and reflectionpaths) may include a primary light source from which some or all of anillumination signal (pulse or otherwise) may be directed toward a windowof the sensing path. In such embodiments, internal reflections from thewindow may become incident upon one or more sensors, and from thisincident light, the blockage analysis (described in more detail below)may be performed. Alternatively or additionally, the light incident onthe at least one sensor used to perform the blockage analysis mayinclude light from one or more light sources ancillary to the primarylight source 1012 of the system 1000. For example, some embodiments mayinclude an ancillary light source located relative to the reception path(e.g., within a housing or compartment associated with a sensor (whichmay be optically independent from a housing or compartment including theprimary light source)). The ancillary light source may be used toilluminate the protective window. Blockage analysis in such embodimentsmay be based, for example, at least partially upon light from theancillary source that is reflected from the protective window and madeincident upon at least one sensor.

Moreover, in some embodiments, the at least one sensor used in theblockage analysis may include sensor 1016 of the system. In some cases,however, one or more additional or alternative sensors may be includedfor receiving incident light used in analyzing blockages on theprotective window. For example, in addition to sensor 1016, which may bepositioned in the light reception path of the system, one or more lightsensitive devices may be included in the transmission path of the systemto monitor light reflections in the transmission path. Such reflectionsmay be used to analyze protective window blockages.

It should be noted that the presently disclosed systems for analyzingprotective window blockages and for taking remedial actions may be basedon any suitable configuration of light sources and light sensors. Insome cases, the disclosed systems for detecting and analyzing windowblockages may be based exclusively upon the light sources and sensorsused by the system 1000 to detect objects in the FOV 1020. In othercases, one or more ancillary sensors and/or one or more ancillary lightsources may be included (in either or both of the transmission path andreception path of relevant LIDAR systems) for detecting and analyzingwindow blockages.

Blockages on a protective window may be identified and/or characterizedbased on any suitable techniques consistent with the disclosed system1000 and/or LIDAR systems. In some embodiments, if an obstruction ispresent, processor 1018 may detect the presence of a particularobstruction and potentially a particular pattern or pattern typeassociated with the detected obstruction on and/or in the protectivewindow 1010. Exemplary obstructions may be caused by salt, mud, roadgrime, snow, rain, dust, bug debris, pollen, bird droppings, or anyother material that may at least partially obscure a protective window1010. In some cases, other exemplary obstructions may be caused byscratches and/or deformations in or on the protective window itself. Inresponse to a detected obstruction, the processor 1018 may initiate ananalysis to determine a type of obstruction. Based on a determined typeof obstruction, processor 1018 may initiate at least one remedial actionbased on the detected obstruction. Some obstructions may also occurbetween the sensor and the protective window (either on components ofthe system 1000 or in the spaces between such components). Thetechniques consistent with the disclosed LIDAR systems may optionally beused, mutatis mutandis, to detect such obstructions and to possibly takeremedial actions based on the detection.

FIG. 10B illustrates an exemplary system 1000 for detecting blockagesand/or impaired sensing including a projecting unit 1028, and a scanningunit 1004, a sensing unit 1006, a processing unit 1008 (not shown).System 1000 may be mounted in a vehicle 1001. For example, system 1000may be mounted on the dashboard of vehicle 1001. In this example, thewindshield 1003 of vehicle 1001 functions as protective window 1010. Insome embodiments, system 1000 may have a protective window 1010 and bepositioned in a vehicle such that the windshield of the vehicle isbetween the light source and FOV.

FIG. 10C is a block diagram representation of processor 1018 includingseveral software modules implemented by processor 1018 consistent withthe present disclosure. For example, among other blocks, processor 1018may implement signal processing module 1026, obstruction classificationmodule 1028, and obstruction clearing module 1030. Modules 1026, 1028,and 1030 may contain software instructions for execution by at least oneprocessor, e.g., processor 1018, included with the system 1000. In someembodiments modules 1026, 1028, and 1030 may be implemented byrespective circuitry, e.g., hardware and/or firmware, instead of or inaddition to software implementations. Signal processing module 1026,obstruction classification module 1028, and obstruction clearing module1030 may cooperate to process reflection signals received by a sensor todetermine the presence of an obstruction, identify characteristicsassociated with detected obstructions, identify reference obstructionpatterns matching the detected obstructions, and classify theobstruction. The modules may also cooperate to cause performance of oneor more processes selected based on the classification to clear theobstruction.

Signal processing module 1026 may receive reflection signals from one ormore sensors and process the signals to extract data from the signalcharacteristics and determine if an obstruction on a protective windowis present. In some embodiments, signal processing module 1026 may beimplemented by processor 1018. In another embodiment, signal processingmodule 1026 may be implemented by a dedicated ASIC or other logiccircuit. Signal processing module 1026 may use any one or combination oftechniques for processing signal information including one or morealgorithms, neural networks, machine learning software, etc. Signalprocessing module 1026 may be the signal processing module which is usedfor the processing of reflections arriving from the FOV of the system, adedicated signal processing module used solely for obstructiondetection, or any other signal processing module of the processor.

Within the context of obstruction detection, the term “reflectionsignals” (which are used for detection of obstruction) is used broadlyand may pertain to any one or more of the following:

-   -   a. Internal reflection signals from the protective window. Such        internal reflection may include internal reflections of light        originating from an internal light source may be used (whether        that source includes the primary LIDAR light source or whether        that source includes any other supplemental light source        internal to LIDAR system 100). The internal reflection signals        may include signals other than light-signals reaching the        protective window from within the LIDAR system and reaching a        matching sensor, such as other types of electromagnetic waves        (e.g., radar signals or others), acoustic waves, and so on.    -   b. External reflections signals originating from a        LIDAR-associated light source (whether located inside the LIDAR        system or not). Such external reflections signals pass through        the protective window. Such external reflections may include        light originating from outside the protective window and passing        into the LIDAR system through the protective window (e.g., where        scratches, water, or other materials on the protective window        cause unexpected external reflections from certain areas of the        protective window). The system may monitor and detect areas of        reduced light transmission (e.g., areas not passing as much        light as neighboring areas). The external reflection signals may        include signals—other than light-signals—which pass through the        protective window into the LIDAR system and reaching a matching        sensor, such as other types of electromagnetic waves (e.g.,        radar signals or others), acoustic waves, and so on.    -   c. Ambient reflections signals arriving from unassociated light        sources (e.g., sun, streetlights, headlights of other vehicles)        or from their reflections by other objects (e.g., sunlight        reflecting off cars, streetlamps light reflecting off buildings        and roads). Such Ambient reflections signals pass through the        protective window. Such ambient reflections may include light        originating from outside the protective window and passing into        the LIDAR system through the protective window (e.g., where        scratches, water, or other materials on the protective window        cause unexpected ambient reflections from certain areas of the        protective window). The system may monitor and detect areas of        reduced light transmission (e.g., areas not passing as much        light as neighboring areas). The ambient reflection signals may        include signals—other than light-signals—which pass through the        protective window into the LIDAR system and reaching a matching        sensor, such as other types of electromagnetic waves (e.g.,        radar signals or others), acoustic waves, and so on.

In some embodiments, if an obstruction is detected, signal processingmodule 1026 may store an obstruction pattern associated with theobstruction. The obstruction pattern of the detected obstruction mayinclude information on obstructed pixels and their spatial orientation.In some embodiments, a particular detected obstruction pattern may beassociated with at least two neighboring instantaneous positions of theat least one light deflector. The particular detected obstructionpattern may be associated with at least two separate instantaneouspositions.

FIGS. 10D and 10E are diagrams illustrating exemplary components andinputs for obstruction classification by the processor 1018 consistentwith disclosed embodiments. In some embodiments, signal processingmodule 1026 may receive image data indicative of an obstruction pattern.Image data indicating a detected obstruction may include informationreceived per-pixel of sensor 1016, for example. With reference to FIGS.10D and 10E, a hit\miss map 1032 of an illustrative five-by-six pixelarray shows obstructing elements detected in four pixels 1034. Theactual field of view of the system may include many more pixels (e.g.1,000, 1,000,000, etc.). In some embodiments, a scattering pattern ofobstructed pixels may be detected in multi-pixel and/or full frameapplications. The decision of blocked items may be done at the pixellevel, at a multi-pixel level (e.g., pertaining to groups of 2×2 or 3×4pixels), or in any other spatial division of the FOV. If the systemincludes a scanning system, the pixels are not necessarily differentpixels of the sensor, but may include information collected from one ormore pixels at different times. For example, the obstruction pattern mayresult from capturing light when the deflector is located in a pluralityof instantaneous positions of the deflector, but corresponding todifferent locations on the protective window (which may be stationary).If the processor is adapted to detect obstruction in internal componentsof the system, similar techniques may be implemented, mutatis mutandis,and the pixels may pertain to locations on the deflector or on any othercomponent (e.g. for detecting scratches or other flaws in a scanningmirror). In some embodiments, the pixels may correspond to pixels of thelight sensors. In some embodiments, each pixel corresponds to a spatialangle in the FOV of the system. For example, in a sensor having only onepixel, the system may scan the FOV to capture a plurality of spatialangles.

In some embodiments, output information of the signal processing module1026 may be used to estimate an ability of the system to detect objectsin the field of view. For example, if an obstruction is detected, theanalyzed signal information may be used to estimate the efficiency,accuracy, and/or ability with which the system 1000 may operate todetect objects in the environment of the LIDAR system 1000 and/orvehicle 1010.

Obstruction classification module 1028 may receive obstructioninformation from signal processing module 1026. Obstruction informationmay include, for example, obstruction pattern, indication of blockeddeflector positions, returned power of the reflection off theobstruction, whether the field of view is partially or fully blocked,temporal information regarding the evolving of obstruction across theFOV, etc. Optionally, obstruction classification module 1028 mayclassify a detected obstruction by analyzing the shape and profile ofthe received signal and/or by analyzing the pixel-by-pixel obstructionpattern created by the obstruction. Based on this analysis, obstructionclassification module 1028 may classify a detected obstruction based onits obstruction pattern to identify a likely type of obstruction, e.g.,mud, pollen, snow, etc. In some embodiments, a single module may performthe functions of signal processing module 1026 and obstructionclassification module 1028.

In some embodiments, obstruction classification module 1028 may comparethe obstruction pattern of the detected obstruction to a referenceobstruction pattern. System 1000 may include an obstruction patterndatabase of obstruction patterns and their respective characterizations.In some embodiments, the obstruction patterns may be stored in memory ofthe system 1000. Obstruction classification module 1028 may employ oneor more techniques for comparing an obstruction pattern with a referenceobstruction pattern and identifying a match.

In some embodiments, the processor 1018 may be configured to apply atleast one pattern recognition algorithm when comparing the detectedobstruction pattern with the reference obstruction patterns. Anexemplary pattern recognition algorithm may include a temporal patternanalysis of the detected obstruction pattern. For example, a patternrecognition algorithm may analyze a reflection signal as shown in FIG.10F during a certain time frame. In another example, the system 1000 maydetect a smear on the protective window from rain drops accumulating onand then running down the protective window.

In another embodiment, the pattern recognition algorithm may include aspatial pattern analysis of the detected obstruction pattern. In someembodiments, the processor 1018 may use model matching 1040 to comparethe detected obstruction pattern 1036 with a known model, e.g., areference obstruction pattern, and may use one or more recognitiontechniques for pattern classification 1042. For example, obstructionclassification module 1028 may apply a pattern recognition algorithm tocompare a hit\miss map, e.g., as shown in FIGS. 10D and 10E, with thespatial data of a reference obstruction pattern. In some embodiments, amatching reference obstruction pattern may be identified using a machinelearning algorithm or a neural network 1044. Exemplary obstructions thatmay be classified via spatial analysis include mud, bird droppings, bugparts, and/or smog residue. An obstruction may cover a single area ofthe protective window, e.g., a chunk of mud, or may be distributedacross the protective window, e.g., a plurality of raindrops.

FIG. 10F includes illustrative examples of obstruction patterns, inaccordance with disclosed embodiments. FIG. 10F illustrates the spatialaspects of the obstruction pattern, but temporal aspects (e.g., the paceand/or order in which areas get obstructed) may also be used forclassifying the obstruction.

As aforementioned, a possible way of detecting obstructions is by usinginternal reflection of light emitted by a light source of the system andreflected back—almost instantaneously—from the protective window. Thismay be implemented in systems in which the illumination assembly isoptically connected to the light detection assembly. Nevertheless, thismay also be implemented in systems in which the illumination assembly isseparated from the light detection assembly, e.g., using a dedicatedlight source for the sensor compartment, or a dedicated sensor for thelight source compartment. FIG. 10G provides light intensity versus timecurves for three types of signals that may be experienced by system1000, including LIDAR system 100. Signal 1046 represents a signalproduced by an internal reflection in the system through a clean part ofthe protective window 1010. The temporal amplitude profile of the signaldepends on various factors such as light source intensity and duration,sensing path sensitivity and temporal behavior, reflectivity of theprotective window, incidence angle of the light onto the protectivewindow, and so on. Signal 1048 represents a signal produced by aninternal reflection in the system through a partially blocked part ofthe protective window. Since the blockage causes more internalreflections from the protective window, the signal is stronger and/orspans over a longer time (e.g., due to the temporal response behavior ofthe sensor), as exemplified in the diagram. Signal 1050 represents asignal generated upon receipt of light that was reflected from a targetobject in a scene and subsequently passed through the protective windowat a later time.

Signal processing module 1026 may receive signal data for a pulse andcompare the received signal to a known internal reflection signal of thesystem. If the comparison is indicative of a different signal, i.e., notan internal reflection signal, the difference in signal may be used todetermine the presence and type of blockage. In some embodiments, thereceived signals and internal reflection signals may be compareddirectly, or by processing parameters of the received signal. Forexample, if the time it takes the received signal to fall below athreshold intensity is longer than predetermined duration, then thereceived signal is indicative of an obstruction. In some embodiments,determining the spatial pattern of a blockage across an area of thewindow/windshield, or temporal blockage across different frames, mayinclude using a dedicated light source on the other side of thewindow/windshield, and determining how much light is transferred towardthe sensor.

For example, in some embodiments, received LIDAR reflections signalsfrom at least one sensor, may include indications of light reflectedfrom the protective window inside the LIDAR system. Signal processingmodule 1026 may determine internal reflection parameters from the LIDARreflections signals and access memory storing signal baseline parametersassociated with the LIDAR system. The system may use the internalreflection parameters and the signal baseline parameters to identify atleast one obstructed portion of the field of view and to alter a lightsource parameter such that more light is projected toward other portionsof the field of view than light projected toward the at least oneobstructed portion of the field of view.

In another example, with reference to FIG. 10G, the system may determinethat a certain pixel is blocked based on a comparison of the receivedsignal and the known internal reflection signal, as described above. Inthis example, the system may not classify the obstruction, but may use(without necessarily classifying any larger obstruction; useful forexample for stopping further illumination to this direction, or on thecontrary—increasing light intensity if possible to shine through partlytransparent obstruction).

Received signal characteristics may include the spatial and/or temporalwidth of a received pulse. Signal processing module 1026 may alsodetermine if an obstruction is present based on one or more thresholdparameters. Such a threshold parameter may include, for example, atemporal parameter, such as the duration of the internal reflection ofan internal reflection signal, its rate of ascend and/or descend, and soon. For example, if a detected signal has a greater duration than thethreshold, signal processing module 1026 may determine that anobstruction is present. In another example, if the intensity of thedetected light signal exceeds a predetermined threshold within apredetermined time window from the emission of the light pulse, and soon. Other parameters such as polarization of the detected signal and soon may also be used. Signal 1050 may be superimposed on signal 1046 ifthere is an external object, but differences such as timing, intensity,polarization, temporal profile, and so on may be used to distinguishbetween reflection resulting from blockage and reflection from anexternal object. In some embodiments, the processor may be configured touse indications of light reflected from objects in the field of view andpassing through the protective window to determine the likelyobstruction-pattern match.

In some embodiments, signal processing module 1026 may analyze one ormore received signals to determine an obstruction pattern created by adetected obstruction. In some embodiments, the processor may beconfigured to detect the particular obstruction pattern based ondifferences between signal baseline parameters and internal reflectionparameters associated with one or more areas (or the entire area) of theprotective window. For example, the signal received from internalreflection 1048 may have profile different from a signal generated by aninternal reflection from an obstruction. In some embodiments, theprocessor may be configured to detect the particular obstruction patternby determining from the reflections signals internal reflectionparameters associated with different areas of the protective window.

In some embodiments, the processor may be configured to detect theparticular obstruction pattern based on light detected during a timeperiod between light leaving the at least one light source andreflection impinging on the least one sensor. For example, rain may bedetected over several frames (e.g. seconds or tens of seconds)—whereobstructions (e.g., raindrops) are accumulating, moving (e.g. slidingdown the window or sideways due to wind), and so on.

In some embodiments, the patterns stored in the memory may be stored inneural network parameters. Patterns may be stored in terms of parametersincluding: physical dimensions of obstructions, aspect ratio, scatteringacross the window, transparency levels, scattering, and so on.

Reflection signals may include any signal or portion of a signaloriginating from a reflection event in a system 1000, e.g., a LIDARsystem 100, RADAR system, or other bi-static system. In some cases, suchreflection events may occur outside the system (e.g., an object in theLIDAR FOV reflecting at least a portion of incident laser light from theLIDAR system). In other cases, such reflection events may be internal tothe LIDAR system or may be associated with a platform upon which a LIDARsystem may be deployed. For example, (e.g., internal light reflectionsfrom one or more LIDAR system components or light reflections from awindshield (e.g., windshield 1003) or other component of a vehicle orother platform upon which a LIDAR system is mounted). LIDAR reflectionssignals may refer to reflected light incident upon one or more LIDARsensors. In other cases, the LIDAR reflections signals may refer toelectrical signals generated by one or more LIDAR sensors in response toincident light originating from one or more reflections events. Suchelectrical signals may be provided to one or more processors for objectdetection analysis, blockage analysis, etc. Contributors to the LIDARreflections signals may include, among other things, light reflectedfrom a LIDAR FOV (e.g., from a detected or to-be-detected object in theFOV), light originating from the scene in the FOV, light from LIDARsystem or LIDAR platform internal reflections, and/or random noiseresulting from various detection path components (sensors, amplifiers,filters, etc.).

After identifying one or more reference obstruction patterns matchingthe detected obstruction pattern, a classification process 1047 mayassign one or more classification characteristics to the detectedobstruction. Characteristics associated with a reference obstructionpattern may include opacity, size, effect on system efficiency, cleaningmethod, etc. In some embodiments, one or more parameters may bedetermined directly from the specific obstruction (e.g., the dimensionsof the mud spot). One or more of these parameters may be stored in thememory and be associated in the class of blockages. In some embodiments,one or more detected parameters may be used for further decisions (e.g.,decision about cleaning procedures timing may be associated with rulesstored in the memory, with knowledge about how much time can you waitbefore cleaning is mandatory, and so on). In some embodiments, theprocessor may be configured to access stored information characterizingreference obstruction patterns for at least four of salt, mud, roadgrime, snow, rain, dust, bug debris, pollen, and bird droppings. Inanother embodiment, the processor may be configured to access storedinformation characterizing reference obstruction patterns for salt andat least one of mud, road grime, snow, rain, dust, bug debris, pollen,and bird droppings. As described previously, characterizationinformation associated with reference obstruction patterns may be storedin a database and/or in a memory of the system 1000.

Obstruction clearing module 1030 may receive obstruction classificationinformation indicating a reference obstruction pattern and associatedcharacteristics for a detected obstruction from obstructionclassification module 1028. Based on the obstruction patterncharacteristics, obstruction clearing module 1030 may output informationinstructing one or more systems within the system 1000 and/or vehicle1001 to execute a remedial action. For example, obstruction clearingmodule 1030 may control activation parameters of one or more cleaningmechanisms, e.g., wipers, washing fluid, condensed air, etc. In anotherembodiment, obstruction clearing module 1030 may alert an operator ofthe vehicle or another system (e.g., host) of the detected obstructionand/or type of obstruction.

In some embodiments, obstruction clearing module 1030 may instruct asystem of the system 1000 and/or vehicle 1001 to execute the remedialaction of cleaning a protective window, e.g., window 1010 or windshield1003. In some embodiments, the remedial action may include a windowcleaning request. For example, the window cleaning request may instructa wiper system to clear the protective window of the system 1000. Insome embodiments, the processor may be configured to output informationthat includes a window cleaning request associated with a determinedcause of the obstruction of the protective window based on theobstruction-pattern match. In some embodiments, the processor may beconfigured to select a cleaning process associated with the determinedcause of the obstruction of the protective window, and to outputinformation that includes a window cleaning request associated with theselected cleaning process. For example, obstruction classificationmodule may classify a detected obstruction as having an obstructionpattern matching dust. Based on this obstruction pattern, obstructionclearing module 1030 may send instructions to a system of the system1000 or vehicle 1001 to spray washing fluid on the protective window andto activate one or more wipers.

In some embodiments, the processor of the system 1000 may be (orinclude) a processor of the host (i.e., higher level system such asvehicle 1001). For example, in a LIDAR system, e.g., LIDAR system 100,processor 118 which executes the distance measurement or otherLIDAR-specific actions may be separate from processor 1018.

In some embodiments, the processor may be configured to cause a changein light flux projected from the at least one light source based on thedetected obstruction pattern and the determined cause of the obstructionof the protective window. For example, obstruction clearing module 1030may execute instructions causing the light source to emit less light toa blocked region. In other example, a remedial action may be to increasethe light emitted if the obstruction is transparent. In someembodiments, the processor may be configured to cause a change in asensitivity of the at least one sensor based on the detected obstructionpattern and the determined cause of the obstruction of the protectivewindow. For example, obstruction clearing module 1030 may executeinstructions to increase the sensitivity of the pixels not obstructed bythe obstruction.

FIG. 11 is a flow chart of an exemplary method 1100 for detecting andclassifying obstructions. As previously described, signal processingmodule 1026 may detect an obstruction (step 1101) by analyzing sensordata 1102. Additionally, sensor data 1102 may be used to generate apoint cloud 1105 as previously described. Data from point cloud 1105 maybe used to determine the obstruction pattern of the detectedobstruction.

In some embodiments, the signal processing part of the LIDAR system 100may output semi-points as part of the point cloud data, indicatingblocked areas of the FOV. For example, instead of outputting a parameterset (θi, ϕi, distance, reflectivity) for a point, the processor mayoutput (θi, ϕi, obstruction (flag), obstruction parameters (if any)). Inanother embodiment, a higher level processing module (such as thecomputer vision processor) may process the point cloud and determine anobstruction. For example, if a given pixel is dark for a predeterminednumber of frames, while surrounding pixels are systematically not dark,the system may define the pixel as obstructed).

Information about the obstruction, including its obstruction pattern maybe sent to obstruction classification module 1028. Using one or morealgorithms, e.g., a temporal or spatial pattern matching algorithm,and/or neural network software, obstruction classification module 1028may match the obstruction pattern with a reference obstruction patternstored in a database 1104 (step 1103). The reference obstruction patternmay contain classification information and characterization informationabout the type of obstruction. Additional input may be used to match thedetected obstruction pattern to a reference obstruction pattern. Forexample, a temperature sensor 1107 of the system 1000 may indicate theambient temperature of the environment of the system is below thefreezing point of water (or any other temperature). In this example,temperature information coupled with a distributed and translucentobstruction pattern may indicate that the obstruction is ice on theprotective window. In another example, a water sensor 1108 of the system1000 may be used classify an obstruction pattern as spattered water, orcontinuous rainfall, which require different remedial actions.

At step 1106, obstruction clearing module 1030 may evaluate availablecleaning resources 1109 based on the obstruction classification. One ormore systems of the system 1000 and/or vehicle 1001 may controlresources available for cleaning the protective window of the system1000 or the windshield 1003 of vehicle 1001. For example, one or moresystems may control a wiper system, washer fluid, de-icing fluid, acompressed air system, defogging system, etc. Clearing module 1030 mayidentify unavailable resources, e.g., a washer fluid container is empty,and alert the operator of the vehicle that the resource is unavailable.In other embodiments, the system may identify other resources that maybe used to clean an obstruction if certain resources are unavailable.For example, obstruction clearing module 1030 may execute instructionsto eject a stream of compressed air at the protective window to removedust if washer fluid is unavailable.

At step 1110, the processor outputs the optimal algorithm for cleaningthe obstruction based on the obstruction pattern and availableresources. At step 1111, the one or more system 1000 or vehicle 1001systems may execute the instructions output by obstruction clearingsystem 1030 to clear the detected obstruction.

At step 1112, the system provides feedback data to the processor. Forexample, feedback data may include instructions to the light source,e.g., not to illuminate towards blocked part of the window. Feedbackdata may further include instructions to one or more system componentsto, for example, correct the point cloud by using software, to alert oneor more system processors that the pointcloud is defective, and/or tocompensate for the blockage using one or more cameras of the vehicle.

At step 1112, the processor 1018 may estimate the efficiency of thesystem to provide feedback. For example, the processor 1018 maydetermine that the detection efficiency increased to a default levelafter a cleaning cycle was executed, indicating that the blockage wascleared effectively. In another example, the system 1000 may continue tooperate at an insufficient efficiency, causing the obstruction clearingmodule to execute instructions to alert a user of the system that thesystem is blocked.

In some embodiments, a cleaning time setting module 1113 may cause acleaning mode setting module 1114 to intermittently execute a defaultcleaning process at predetermined time intervals. For example, module1114 may receive instructions to execute a window cleaning process onceevery ten minutes as dictated by the allotted downtime 1115 and timingmodule 1113. In other embodiments, module 1114 may receive instructionsto decrease the time between cleaning cycles if, for example, blockagedata indicates frequent blockages. In one example, a vehicle 1001 may bepassing by a dusty construction site. Upon detection of frequentblockages, the system may decrease the time between cleaning cycles,such that the dust is cleared from the system window more frequently.

FIG. 12 is a flowchart of an exemplary process 1200 for detecting andcleaning an obstruction consistent with the closed embodiments.Referring to the examples set forth with respect to the previousdrawings, process 1200 may be executed by processor 118 and/or processor1018. At step 1201 the processor 1018 controls at least one lightsource, e.g., light source 1012, to emit light in the direction of thefield of view. As previously described, the system 1000 may control anadditional light source to emit light to detect obstructions.

At step 1202, the processor receives reflections signals from at leastone sensor, e.g., sensor 1016. Reflections signals may be analyzed bysignal processing module 1026 as previously described. Reflectionssignals may include indications of light reflected from the protectivewindow and light reflected from objects in the field of view and passingthrough the protective window prior to reaching the at least one sensor.

At step 1203, the processor may detect a particular obstruction patternat least partially obstructing light passing through the protectivewindow of the system 1000. These reflections signals may includeindications internal reflections of light from an internal light sourceoff the protective window; (b) external reflections of light from aninternal light source; and (c) ambient light. An obstruction may be anyobject or defect blocking the protective window, e.g., salt, mud, roadgrime, snow, rain, dust, bug debris, pollen, condensation, ice, and birddroppings, or a defect or scratch in the protective window.

At step 1204, the processor may access stored information characterizingreference obstruction patterns for at least one of salt, mud, roadgrime, snow, rain, dust, bug debris, pollen, and bird droppings.Exemplary obstruction patterns may include those shown in FIG. 10F. Forexample, obstruction classification module 1028 may access a database1104, or other memory device to retrieve one or more referenceobstruction patterns corresponding to one or more types of possibleobstruction.

At step 1205, the processor may compare the detected obstruction patternwith the reference obstruction patterns in order to determine a likelyobstruction-pattern match. As previously described, obstructionclassification module 1028 may use one or more pattern matchingalgorithms and/or neural network or machine learning software to matchthe detected obstruction pattern to a reference obstruction pattern.

At step 1206, the processor may output information indicative of thematch based on the likely match. Parameters of the matching referenceobstruction pattern may be output to obstruction clearing module 1030and used to determine a remedial action for clearing the detectedobstruction. In some embodiments, reference obstruction pattern datareceived from a database storing reference obstruction patterns mayinclude an optimal cleaning algorithm. The processor may further outputinformation to alert vehicle operator that an obstruction was detected.

In some embodiments, output may include obstruction location (e.g. withrespect to the window/windshield), obstruction location with respect tothe point cloud (e.g., for changing illumination levels), parameters ofthe obstruction (e.g., classification, dimensions, time-stamp, temporalprofile, transparency), raw data (detected signals), indication ofblocked pixels, and so on.

Aggregating Pixel Data Associated with Multiple Distances or AggregatingOver Time to Improve Image Quality

Signal Aggregation Across Distances

In some embodiments, LIDAR systems of the present disclosure mayaggregate data across different distances to improve image quality. Forexample, the aggregation may improve spatial resolution, distanceestimates, surface angle determinations, object classifications, or anyother measurements performed by the LIDAR system. In addition, theaggregation may provide for more accurate processing of split field ofview pixels. Split FOV pixels are those including both foreground andbackground information. As used herein, “foreground” may refer toportions of a field of view, such as objects, road markings, or thelike, that are within a particular distance (whether measured inCartesian coordinates or spherical coordinates) of a light source and/ora detector of the LIDAR system while “background” may refer to portionsof the field of view, such as objects, road markings, or the like, thatare beyond the particular distance of the light source and/or thedetector of the LIDAR system. Alternatively, any object, road marking,or the like may be classified as “foreground” with reference to anotherobject, road marking, or the like that is farther from the light sourceand/or the detector of the LIDAR system than the foreground object andthus classified as “background.”

In some embodiments, the classification of an FOV pixel as “background,”“foreground,” or “split” may be explicit. For example, the explicitclassification may be based on determined distances based on signalsfrom the pixel caused by received reflections from the field of view.Alternatively, the classification of a pixel as “background,”“foreground,” or “split” may be an implicit consequence of implementinga binning scheme. For example, a group of avalanche photodiodes maygenerate a plurality of input signals indicative of reflections of lightfrom objects in the field of view, and the plurality of input signalsmay be placed in a plurality of bins, each bin corresponding to inputsignals caused by reflections from a region of the field of view.Accordingly, pixels may be “background,” “foreground,” or “split” as aconsequence of binning and the objects, road markings, or the likepresent within the regions corresponding to the bins rather than basedon an explicit classification.

In some embodiments, one or more FOV pixels may fall outside these threecategories. For example, some pixels may be “intermediate pixels” whoseassociated object distances from the LIDAR system are longer than thoseassociated with the “foreground” pixels, and shorter than thoseassociated with the “background” pixels. As mentioned above, differentdefinitions of “background,” “foreground,” and “intermediate” may beused in different implementations of the invention, and possibly by asingle LIDAR system at different times or settings. For example,foreground objects may include objects closer to the LIDAR system than afirst threshold distance such as 20 m, 30 m, 40 m, 50 m, or the like.For example, background objects may include objects further from theLIDAR system than a second threshold distance such as 40 m, 50 m, 75 m,100 m, or the like. In a given system, the second threshold distance maybe equal or larger than the first threshold distance.

FIG. 13A is a diagram illustrating an exemplary LIDAR system 1300 havinga plurality of pixels. As depicted in FIG. 13A, the LIDAR system 1300may include projecting unit 1302, which may include at least one lightsource 1312; scanning unit 1304, which may include at least one lightdeflector 1314 and optional optical window 1324; and sensing unit 1306,which may include at least one sensor 1316. As further depicted in FIG.13A, the field of view 1322 may include a plurality of pixels, e.g.,pixels 1322 a to 1322 d. Accordingly, each “pixel” may comprise aportion of the field of view.

FIG. 13B is a diagram illustrating an implementation 1350 of LIDARsystem 1300 of FIG. 13A. The implementation 1350 of FIG. 13B is ascanning and monostatic LIDAR; however, any other LIDAR system, such asnon-scanning systems, multistatic systems, or the like, may be used toimplement the embodiments described below. The example LIDAR system 1350of FIG. 13B may include projecting unit 1302 (which may, for example, besimilar to projecting unit 102 of FIG. 1A, 2A, 2B, or 2C, describedabove), which may include at least one light source 1312 (which may, forexample, be similar to light source 112 of FIG. 1A, 2A, or 2B, describedabove). The example system 1350 of FIG. 13B may further include scanningunit 1304 (which may, for example, be similar to scanning unit 104 ofFIG. 1A, 2A, 2B, or 2C described above), which may include at least oneoutgoing light deflector 1314 and at least one incoming light deflector1326 (which may, for example, be similar to deflectors 114A and 114B ofFIG. 2A described above); however, in alternate implementations, one ormore deflectors may be used for both outgoing and incomingtransmissions, such as deflector 114 of FIG. 2B, described above. Theexample system 1350 of FIG. 13B may further include sensing unit 1306(which may, for example, be similar to sensing unit 106 of FIG. 1A or2A, described above), which may include at least one sensor 1316 (whichmay, for example, be similar to sensor 116 of FIG. 1A, 2A, 2B, or 2Cdescribed above). Sensor 1316 may include a plurality of detectionelements, e.g., elements 1316 a, 1316 b, and 1316 c. As explained above,each pixel in field of view 1322 may be detected by a plurality ofdetection elements, such as Avalanche Photo Diodes (APD), Single PhotonAvalanche Diodes (SPADs), combination of Avalanche Photo Diodes (APD)and Single Photon Avalanche Diodes (SPADs), or other detecting elementsthat measure both the time of flight from a laser pulse transmissionevent to the reception event and the intensity of the received photons.The outputs of the detection elements corresponding to each FOV pixelmay be summed, averaged, or otherwise combined to provide a unifiedoutput.

Accordingly, as used herein, the terms “pixel” or “FOV pixel” refer toany three-dimensional portion of the field of view (e.g., defined byfour Cartesian values such as (x+dx, y+dy), (x±dx, y±dy), or the like ordefined by four angular values such as (0+dθ, ϕ+dϕ), (θ±dθ, ϕ±dϕ), orthe like). Each pixel's size (e.g., determined by dx and dy or by dθ anddϕ) may be the same or may differ. Some pixels may be mutually exclusive(i.e., non-overlapping) while other pixels may at least partiallyoverlap. Some pixels may be defined by adjacent portions of the detector(e.g., at least one sensor 1316) and/or by adjacent detection elementson the detector while other pixels may be associated withnon-neighboring detection elements. As referenced elsewhere, an FOVpixel refers to a region of the field of view from which one or morereceived reflections signals are combined together to generate a singledata point within the point cloud generated by a LIDAR system.

As depicted in FIG. 13B, light source 1312 may emit one or more lightbeams (or groups of light beams), e.g., projected beams 1328 a, 1328 b,and 1328 c, toward field of view 1322. The projected beams may reflectoff objects, road markings, or the like, in field of view 1322, causingcorresponding one or more reflected light beams (or groups of lightbeams), e.g., reflected beams 1330 a, 1330 b, and 1330 c, to travel backtowards the LIDAR system. As further depicted in FIG. 13B, eachprojected beam may be associated with a particular region of field ofview 1322. Accordingly, each reflected beam may also be associated witha corresponding region.

Furthermore, each pixel may be associated with the region correspondingto the region of field of view 412 from which the one or more reflectedbeams received on the pixel originate. For example, as depicted in theexample of FIG. 13, detection element 1316 a receives reflected beam1330 a caused by projected beam 1328 a, detection element 1316 breceives reflected beam 1330 b caused by projected beam 1328 b, anddetection element 1316 c receives reflected beam 1330 c caused byprojected beam 1328 c. Although depicted as a one-to-one correspondencein FIG. 13B, one or more of the projected light beams may illuminate aplurality of pixels in the field of view 1322. Moreover, as explained infurther detail below, a processing unit (not depicted), which mayinclude at least one processor, may aggregate data across pixelsassociated with different regions to improve accuracy, resolution, orthe like.

FIG. 14A is is a diagram illustrating an exemplary LIDAR system 1400 forcollecting information from at least three pixels in the field of view.In the example of FIG. 14A, system 1400 has at least a field of viewwith a first pixel 1401 a, a second pixel 1401 b, and a third pixel 1401c. As further depicted in FIG. 14B, first pixel 1401 a includes aportion of a foreground object and an intermediate object, second pixel1401 b includes a portion of the intermediate object and a backgroundobject, and third pixel 1401 c includes the background object.Accordingly, first pixel 1401 a may be classified as a foreground pixeland/or a split pixel, second pixel 1401 b may be classified as anintermediate pixel and/or a split pixel, and third pixel 1401 c may beclassified as a background pixel. FIG. 14B is a diagram illustrating afront-facing view of the pixels of FIG. 14A.

FIG. 14C is a diagram illustrating an exemplary vehicle with a LIDARsystem 1400. In the example of FIG. 14C, the field of view of system1400 has at least three pixels. For example, system 1400 may comprisesystem 1300 of FIGS. 13A and/or 13B, described above. As depicted inFIG. 14C, system 1400 may output data corresponding to at least threepixels associated with regions of a field of view, e.g., regions 1401,1402, and 1403. Each of these pixels may be associated with one or moreforeground objects, one or more background objects, or divided betweenone or more foreground objects and one or more background objects. Forexample, as shown in the example of FIG. 14D, foreground object 1406 maybe classified as “foreground” because it is within a threshold distanceof the LIDAR system 1400 and background object 1408 may be classified as“background” because it is beyond the threshold distance. In the exampleof FIG. 14D, region 1402 includes foreground object 1406, region 1404includes background object 1408, and region 1403 is split betweenforeground object 1406 and background object 1408. Although depictedwith three pixels, LIDAR system 1400 may detect more than three pixelsin its FOV, and may have varying numbers of any of the pixels typesdescribed above (e.g., described for regions 1402, 1403, and 1404). Insome embodiments, each pixel of the LIDAR system may detect differentobjects in different times, and therefore the classification of pixelsto such categories may also change with time. As described above, someof the pixels of the LIDAR system may not fall into these categories(such as pixels with no detected image, malfunctioning or erroneousequipment output relative to certain pixels, and pixels detectingobjects with intermediary distances).

In another example depicted in FIG. 14E, foreground object 1406′ may atleast partially occlude background object 1408. Furthermore, as shown inthe example of FIG. 14E, foreground object 1406′ may be classified as“foreground” because it is closer to LIDAR system 1400 as compared tobackground object 1408. In the example of FIG. 14E, region 1402 includesforeground object 1406′, region 1404 includes background object 1408,and region 1403 is split between foreground object 1406′ and backgroundobject 1408.

In any of the examples of FIGS. 14C, 14D, and 14E, signals associatedwith region 1403 (e.g., corresponding to one or more pixels comprisingoutput based on reflections from region 1403) may be aggregated withsignals associated with region 1404 (e.g., corresponding to one or morepixels comprising output based on region 1404). This aggregation mayimprove signal accuracy, resolution, or the like for signals related tobackground object 1408. Additionally or alternatively, signalsassociated with region 1403 (e.g., corresponding to one or more pixelscomprising output based on region 1403) may be aggregated with signalsassociated with region 1402 (e.g., corresponding to one or more pixelscomprising output based on region 1402). This aggregation may improvesignal accuracy, resolution, or the like for signals related toforeground object 1406 (or foreground object 1406′).

Although the examples of FIGS. 14C, 14D, and 14E depict three pixels,additional pixels may be used. For example, as depicted in the exampleof FIG. 14F, region 1402 includes foreground object 1406, region 1404and split region 1410 include background object 1408′, and region 1403is between foreground object 1406 and background object 1408′.Accordingly, similar to FIGS. 14C, 14D, and 14E, region 1403 (e.g.,corresponding to one or more pixels comprising output based on region1403) may be aggregated with signals associated with region 1404 (e.g.,corresponding to one or more pixels comprising output based on region1404) and/or region 1410 (e.g., corresponding to one or more pixelscomprising output based on region 1410). This aggregation may improvesignal accuracy, resolution, or the like for signals related tobackground object 1408′. Additionally or alternatively, signalsassociated with region 1403 (e.g., corresponding to one or more pixelscomprising output based on region 1403) may be aggregated with signalsassociated with region 1402 (e.g., corresponding to one or more pixelscomprising output based on region 1402). This aggregation may improvesignal accuracy, resolution, or the like for signals related toforeground object 1406.

FIG. 15 is a flowchart of method 1500 for using LIDAR to determinedistances to objects in a field of view. For example, method 1500 may beimplemented by at least one processor of a LIDAR system (e.g., at leastone processor 1318 of LIDAR system 1300, at least one processor 118 ofLIDAR system 100, or the like) and/or by at least one processor within abody of a vehicle (e.g., processor 408 of housing 200B of vehicle 110).

At step 1501, the at least one processor may control at least one LIDARlight source in a manner enabling light flux to vary over a plurality ofscans of a field of view. For example, the at least one processor mayvary the timing of pulses from the at least one light source.Alternatively or concurrently, the at least one processor may vary thelength of pulses from the at least one light source. By way of furtherexample, the at least one processor may alternatively or concurrentlyvary a size (e.g., length or width or otherwise alter a cross-sectionalarea) of pulses from the at least one light source. In a yet furtherexample, the at least one processor may alternatively or concurrentlyvary the amplitude and/or frequency of pulses from the at least onelight source. In certain aspects, the at least one processor may varythe light flux during a single scan and/or across a plurality of scans.Additionally or alternatively, the at least one processor may vary thelight flux across a plurality of regions in the field of view (e.g.,during a scan and/or across a plurality of scans).

In some embodiments, method 1500 may further include controlling atleast one light deflector to deflect light from the at least one lightsource such that during a single scanning cycle the at least one lightdeflector instantaneously assumes a plurality of instantaneouspositions. In one example, the at least one processor may coordinate theat least one light deflector and the at least one light source such thatwhen the at least one light deflector assumes a particular instantaneousposition, a portion of a light beam is deflected by the at least onelight deflector from the at least one light source towards an object inthe field of view, and reflections of the portion of the light beam fromthe object are deflected by the at least one light deflector toward atleast one sensor. In another example, the at least one light source maycomprise a plurality of lights sources aimed at the at least one lightdeflector, and the at least one processor may control the at least onelight deflector such that when the at least one light deflector assumesa particular instantaneous position, light from the plurality of lightsources is projected towards a plurality of independent regions in thefield of view.

In such embodiments, for each instantaneous position of the at least onelight deflector, the group of detectors may be configured to generateinput signals corresponding to an instantaneous portion of the field ofview. For example, the input signals may be explicitly associated withthe instantaneous position (e.g., by adding an address or otherindicator to the signal) or may be implicitly associated (e.g., by beinggenerated within a time frame that associates them to a particularinstantaneous position).

Additionally or alternatively, the generated input signals correspondingwith the instantaneous position of the at least one light deflector maybe associated with a plurality of pixels. For example, the input signalsmay be explicitly associated with the pixels (e.g., by adding an addressor other indicator to the signal) or may be implicitly associated (e.g.,by being generated by detectors associated with the pixels).

In other embodiments, method 1500 may be performed without varying thelight flux of the at least one light source. For example, method 1500may be performed with a LIDAR system that is fixed-power rather thanvariable-power.

In some embodiments, the at least one processor may further control theat least one light source in a manner enabling modulating the projectedlight and to distinguish between light reflected from objects in thefield of view and light emitted by objects in the field of view. Forexample, the at least one processor may pulse the at least one lightsource such that the gap between pulses is sufficiently long to receivelight emitted by objects in the field of view rather than lightreflected from objects in the field of view.

The field of view may include a foreground area and a background area.As explained above, the foreground area may include a portion of thefield of view within a threshold distance of the LIDAR system (or acomponent thereof). Accordingly, the background area may include anotherportion of the field of view outside the threshold distance. In someembodiments, different thresholds may be used by the LIDAR system atdifferent times and/or in different parts of the field of view. Forexample, the threshold may be determined based on operational parameters(e.g., illumination level, sensitivity of sensor), on environmentalcharacteristics (e.g., sky, road, vehicles, vegetation), or the like.Alternatively, the foreground area and the background area may bedefined with respect to relationships between objects, road markings, orthe like in the field of view. For example, an object may be designatedas a “foreground” object if it is closer to the LIDAR system (or acomponent thereof) than a reference object designated as a “background”object or vice versa.

At step 1503, the at least one processor may receive from at least onedetector a plurality of input signals indicative of light reflected fromthe field of view. For example, the at least one processor may receivesignals from at least one detector associated with a first pixel.

A representation of a portion of the field of view associated with aplurality of pixels may be constructible from the plurality of inputsignals. For example, a point cloud map may be constructible usingdistances determined based on the input signals. In such an example, theat least one processor could generate a point cloud map by determiningtime-of-flight for one or more of the input signals, determiningdistances based on times-of-flight, and generating the point cloud mapbased on the determined distances.

The plurality of input signals may be associated with a first pixel thatcovers a portion of the foreground area, a second pixel that covers aportion of the foreground area and a portion of the background area, anda third pixel that covers a portion of the background area. As explainedabove, “background” and “foreground” may be defined using a distancethreshold or using relative distances. For example, the “backgroundarea” may simply include one or more points that are further from theLIDAR system (or a component thereof) than one or more points in the“foreground area.” In addition, as explained above, each pixel mayinclude a plurality of detectors, each detector generating one or moreinput signals. Furthermore, as explained above, the first pixel, thesecond pixel, and the third pixel may be non-overlapping or may at leastpartially overlap. In some embodiments, any two or more of the firstpixel, the second pixel, and the third pixel may be adjacent to eachother.

In some embodiments, the at least one processor may identify the firstpixel that covers a portion of the foreground area, the second pixelthat covers a portion of the foreground area and a portion of thebackground area, and the third pixel that covers a portion of thebackground area. For example, the at least one processor may usetime-of-flight to identify the first pixel, the second pixel, and thethird pixel, the time-of-flight for at least some signals associatedwith the first pixel being shorter than the time-of-flight for at leastsome signals associated with the second pixel and the time-of-flight forat least some signals associated with the third pixel.

Alternatively, the at least one processor may not explicitly identifythe first pixel, the second pixel, and the third pixel. For example, theat least one processor may receive, from a group of avalanchephotodiodes the plurality of input signals indicative of reflections oflight from objects in the field of view and then process a first subsetof the input signals associated with a first region and process a secondsubset of the input signals associated with a first region. Byprocessing the signals using this or similar binning techniques, the atleast one processor may divide the plurality of input signals into thefirst pixel, the second pixel, and the third pixel without explicitidentification of the FOV pixels themselves.

Although described above and below using three pixels, a larger numberof pixels may also be used. For example, in embodiments employing a 2×2binning scheme, a plurality of “third” pixels may be used. Accordingly,any aggregation described below may include additional pixels covering aportion of the background area.

In embodiments employing binning, the at least one processor may bin theinput signals in a plurality of ways. Accordingly, one binning schememay produce a first pixel, a second pixel, and a third pixel (e.g.,groupings of input signals combined together to generate a single datapoint in the point cloud output) such that aggregating the second pixeland the third pixel does not result in an object-existence-certaintylevel above a threshold while another binning scheme may produce adifferent first pixel, second pixel, and third pixel such thataggregating the second pixel and the third pixel does result in anobject-existence-certainty level above a threshold.

As explained above, each pixel may correspond to a plurality ofdetection elements, such as a plurality of SPADs, which may beaggregated to a single SiPM. Such aggregation is not considered binningbecause it is implemented via a dedicated hardware architecture intendedto yield the input signals (e.g., a single output signal for an SiPM atany given moment) rather than being implemented on a higher level (e.g.,using the at least one processor) and being performed on the producedinput signals (e.g., combining the concurrent outputs of two or moreSiPMs).

As explained above input signals associated with the second pixel may begenerated in response to a plurality of reflections impinging on thegroup of detectors. Moreover, the plurality of reflections may include afirst reflection associated with a first time-of-flight and a secondreflection associated with a second time-of-flight longer than the firsttime-of-flight. Additionally or alternatively, the plurality reflectionsmay include a first reflection associated with a first directionrelative to the at least one light source and a second reflectionassociated with a second direction relative to the at least one lightsource.

At step 1505, the at least one processor may use input signalsassociated with the first pixel to determine a distance to a firstobject located in the foreground area. For example, as explained above,time-of-flight of at least some signals associated with the first pixelmay be used to determine the distance.

At step 1507, the at least one processor may use input signalsassociated with the second pixel and input signals associated with thethird pixel to determine a distance to a second object located in thebackground area. For example, as explained above, time-of-flight of atleast some signals associated with the second pixel and time-of-flightof at least some signals associated with the third pixel may be used todetermine the distance. The times-of-flight from the second pixel andthe third pixel may be combined prior to determining the distance and/ordistances based on the second pixel and third pixel may be determinedand then combined.

In some embodiments, the input signals associated with the second pixeland the input signals associated with the third pixel may beinsufficient by themselves to determine a distance to the second objectlocated in the background area. Accordingly, as explained above, the atleast one processor may combine the input signals to determine thedistance.

In some embodiments, step 1507 may further include initially processingonly input signals associated with the third pixel. When a certaintylevel associated with an existence of the second object is below athreshold, the at least one processor may combine input signalsassociated with the second pixel and input signals associated with thethird pixel to improve the certainty level associated with the existenceof the second object. Accordingly, the certainty levels for a presenceof an object in either the second pixel or the third pixel may be belowa detection threshold, but the combining of both signals may allowdetection of the second object by raising the certainty level above thedetection threshold.

In any of the embodiments discussed above, a Signal to Noise Ratio (SNR)associated with the third pixel may be lower than an SNR associated withthe second pixel. In any of the embodiments discussed above, the secondpixel may be adjacent to the third pixel. In such embodiments, the atleast one processor may assign a same distance to the second pixel andthe third pixel. Alternatively, the second pixel may be non-adjacent tothe third pixel.

In any of the embodiments described above, the first pixel may beassociated with a first number of detectors, the second first pixel maybe associated with a second number of detectors, and the third pixel maybe associated with a third number of detectors. For example, the firstpixel may be composed of input signals from four detectors, the secondfirst pixel may be composed of input signals from eight detectors, andthe third pixel may be composed of input signals from twelve detectors.Accordingly, the at least one processor may assign differing numbers ofdetectors out of the group of detectors to the second pixel and to thethird pixel. The assignment may be explicit or a consequence of theapplication of one or more binning schemes, as discussed above.

Method 1500 may further include additional steps. For example, method1500 may further include determining a direction for the second objectis based on directions of the second pixel and the third pixel. In suchembodiments, the at least one processor may generate a point cloud andinclude a direction that is a combination of the direction based on thesecond pixel and the direction based on the third pixel (e.g., anaverage of these directions). Similar to the distance determine in step1507, the direction based on the second pixel and the direction based onthe third pixel may be combined prior to determining the combineddirection and/or directions based on the direction of the second pixeland the direction of the third pixel may be determined and thencombined.

Method 1500 may further include using input signals associated with thesecond pixel and input signals associated with the first pixel todetermine a distance to the first object located in the foreground area.Accordingly, aggregation may be used to improve distance measurementsfor a foreground object in addition to a background object.

In addition to determining the distance, method 1500 may further includeusing the input signals associated with the first pixel to determine atleast one of: a velocity of the first object, a surface angle of thefirst object, a reflectivity level of the first object, and ambientlight associated with the first object. Additionally or alternatively,method 1500 may further include using the input signals associated withthe second pixel and the third pixel to determine at least one of: avelocity of the second object, a surface angle of the second object, areflectivity level of the second object, and ambient light associatedwith the second object.

In any of the embodiments described above, method 1500 may includeoutputting information associated with the first object located in theforeground area and/or outputting information associated with the secondobject located in the background area. The output information may beused in further determinations (such as surface angle determination,point cloud generation, or the like) and/or stored for future use.

As explained above, any number of pixels may be used, even though theexample of method 1500 uses three. For example, the at least oneprocessor may identify (whether explicitly or implicitly using binning,as explained above) a fourth pixel. The fourth pixel may be adjacent ornon-adjacent to the third pixel and may cover another portion of thebackground area. In such an example, similar to step 1507, the at leastone processor may use input signals associated with the second pixel,the third pixel, and the fourth pixel to determine a distance to thesecond object located in the background area. The number of additionalpixels used for the determination of the distance to the second objectmay be determined in advance and/or dynamically determined based on thedetection results themselves (e.g., of one or more of pixels and/oraggregated pixels).

Signal Aggregation Across Time

Additionally or alternatively to aggregation across distances, asexplained above, LIDAR systems of the present disclosure may aggregatesignals across time. For example, the aggregation may improve detectionrate, spatial resolution, distance estimates, surface angledeterminations, object classifications, or any other measurementsperformed by the LIDAR system. In some embodiments, the aggregation maybe performed only when one of more of the received signals fall belowone or more thresholds (e.g., brightness threshold(s), confidencethreshold(s), or the like). Additionally or alternatively, a cascade ofaggregations may be applied until the aggregated signal exceeds athreshold (e.g., a brightness threshold, a confidence threshold, or thelike).

Although described below with respect to signal strength and confidence,any measurement derived from the received signals (and/or a confidencescore associated with the measurement) may be aggregated. Accordingly,any measurement (and/or a confidence score associated with themeasurement) may be compared to one or more thresholds to determine whento aggregate and/or when to terminate a cascading aggregation.

FIG. 16 is a diagram illustrating an exemplary vehicle 110 with a LIDARsystem using at least two scan cycles to detect objects. For example,vehicle 110 may include the system of FIG. 13B, described above. Vehicle110 may include at least two pixels associated with regions of a fieldof view. Moreover, vehicle 110 may scan the associated regions over thecourse of multiple scan cycles, e.g., scanning regions 1601 and 1603 inthe first scan cycle and scanning regions 1601′ and 1603′ in the secondscan cycle. Vehicle 110 may receive signals associated with objects,road markings, or the like during scan cycles, e.g., object 1605.

In the example of FIG. 16, signals associated with the first scan cyclemay be aggregated with signals associated with the second scan cycle.This aggregation may improve signal accuracy, resolution, or the likefor signals related to object 1605. Additionally, signals from the firstscan cycle associated with one region may be aggregated with signalsfrom the second scan cycle associated with the same region, e.g.,signals associated with region 1601 combined with signals associatedwith region 1601′, signals associated with region 1603 combined withsignals associated with region 1603′, or the like. This aggregation mayimprove signal accuracy, resolution, or the like for signals related toobject 1605. Although the example of FIG. 16 depicts that the aggregatedregions in the first cycle and in the second cycle do not overlap, theseregions may overlap in other embodiments not depicted (e.g., if both theLIDAR system and the detected objects are stationary, or if both move inthe same speed).

FIG. 17A is a diagram illustrating adjusting an operating parameterbetween scan cycles. In example A of FIG. 17A, a confidence levelassociated with a reflection caused by a pulse from the first cycle of aLIDAR scan is below a threshold. Although depicted as a confidence levelassociated with the reflection itself, the confidence level could beassociated with any detection (such as confidence of the presence of anobject, also called an “object-existence-certainty level”) and/ormeasurement based on the reflection (such a surface angle, reflectivity,object surface composition, or the like. In example A, because theconfidence level is below the threshold, the LIDAR system adjusts acorresponding pulse (e.g., projected towards the same area, object, orthe like as the pulse from the first scan cycle) in the second scancycle to have an increased amplitude.

Similarly, in example B of FIG. 17A, because the confidence level isbelow the threshold, the LIDAR system adjusts a sensitivity of one ormore corresponding detectors (e.g., detectors that received the pulsefrom the first scan cycle) in the second scan cycle to have an increasedsensitivity. The sensitivity may be increased by adjusting a parameterof the detector itself (e.g., decreasing a threshold below whichdetection does not occur) and/or by adjusting a parameter of at leastone processor of the LIDAR system (e.g., decreasing a threshold belowwhich a measurement is disregarded). Additionally or alternatively,sensitivity may be increased by adjusting a parameter along thedetection path, such as increasing the level of amplification ofreceived signals before transmission to the at least one processor.

FIG. 17B is a diagram illustrating using a plurality of thresholds totrigger signal aggregation. In the example of FIG. 17B, a confidencelevel, such as an object-existence-certainty level or other measurementdescribed above with reference to FIG. 17A, associated with a reflectioncaused by a pulse from the first cycle of a LIDAR scan is below a firstthreshold but above a second threshold. Similarly, a confidence level,such as an object-existence-certainty level or other measurementdescribed above with reference to FIG. 17A, associated with a reflectioncaused by a pulse from the second cycle of a LIDAR scan is also belowthe first threshold but above the second threshold. The second thresholdmay ensure that noise and other insignificant signals are notaggregated, which may increase accuracy and conserve valuable processingresources. Because the confidence levels are below the first thresholdbut above the second threshold, the LIDAR system combines the firstsignal and the second signal to generate an aggregated signal. Asdepicted in FIG. 17B, the aggregated signal may exceed the firstthreshold.

FIG. 17C is a diagram illustrating using a threshold to trigger furthersignal aggregation. In the example of FIG. 17C, a cascading aggregationmay be applied. For example, as depicted in FIG. 17C, a confidencelevel, such as an object-existence-certainty level or other measurementdescribed above with reference to FIG. 17A, associated with a reflectioncaused by a pulse from the first cycle of a LIDAR scan is below athreshold. In addition, a confidence level, such as anobject-existence-certainty level or other measurement described abovewith reference to FIG. 17A, associated with a reflection caused by apulse from the second cycle of a LIDAR scan is also below the threshold.Because the confidence levels are below the threshold, the LIDAR systemcombines the first signal and the second signal to generate anaggregated signal. Moreover, the aggregated signal (comprising thereflection caused by the pulse from the first cycle and the reflectioncaused by the pulse from the second cycle) is also below the threshold.Accordingly, the LIDAR system combines a third signal (comprising areflection caused by a pulse from the third cycle of a LIDAR scan) withthe aggregated signal to generate a second aggregated signal. Asdepicted in FIG. 17C, the second aggregated signal may exceed thethreshold.

Although depicted using three scan cycles, any number of scan cycles maybe used. For example, the LIDAR system may continue to aggregate signalsfrom further scan cycles until the most recent aggregated signal exceedsthe threshold. In such embodiments, a cap on the number of aggregationsmay be implemented. For example, the aggregation may be disregarded ifthe threshold is not exceeded after three, four, five, ten, or the likescan cycles.

Although depicted separately, the cascading example of FIG. 17C may becombined with the two thresholds of FIG. 17B. For example, signals maybe required to be above the second threshold to be included in thecascading aggregation. Additionally or alternatively, the cascadingaggregation may not be started and/or may be terminated when one or moresignals below the second threshold are received.

FIG. 17D is a diagram illustrating using further signal aggregation toimprove resolution. In the example of FIG. 17D, an alternative cascadingaggregation may be applied. For example, as depicted in FIG. 17D, aconfidence level, such as an object-existence-certainty level or othermeasurement described above with reference to FIG. 17A, associated witha reflection caused by a pulse from the first cycle of a LIDAR scan isbelow a threshold. In addition, a confidence level, such as anobject-existence-certainty level or other measurement described abovewith reference to FIG. 17A, associated with a reflection caused by apulse from the second cycle of a LIDAR scan is also below the threshold.Because the confidence levels are below the threshold, the LIDAR systemcombines the first signal and the second signal to generate anaggregated signal. Moreover, the aggregated signal (comprising thereflection caused by the pulse from the first cycle and the reflectioncaused by the pulse from the second cycle) is above the threshold.Accordingly, in order to improve the point resolution of the aggregatedsignal and/or to increase the accuracy of a measurement based on theaggregated signal, the LIDAR system combines a third signal (comprisinga reflection caused by a pulse from the third cycle of a LIDAR scan)with the aggregated signal to generate a second aggregated signal (notshown). A point resolution of the second aggregated signal and/or theaccuracy of a measurement determined based on the second aggregatedsignal may be increased compared to that of the aggregated signal.

Although depicted using three scan cycles, any number of scan cycles maybe used. For example, the LIDAR system may continue to aggregate signalsfrom further scan cycles until the point resolution of or accuracy of ameasurement (e.g., based on an associated confidence level) based on themost recent aggregated signal exceeds a threshold. In such embodiments,a cap on the number of aggregations may be implemented. For example, theaggregation may be halted if the threshold is not exceeded after three,four, five, ten, or the like scan cycles.

Although depicted separately, the cascading example of FIG. 17D may becombined with the two thresholds of FIG. 17B. For example, signals maybe required to be above the second threshold to be included in thecascading aggregation. Additionally or alternatively, the cascadingaggregation may not be started and/or may be halted when one or moresignals below the second threshold are received.

Although the examples of FIGS. 17A, 17B, 17C, and 17D all use staticthresholds, any of the thresholds depicted may be dynamic. In someembodiments, the threshold for one scan cycle may be different thananother scan cycle. For example, the threshold may be adjusted based onan amplitude of the projected light during the scan cycle, based on alevel of ambient light received during (or immediately before and/orafter) the scan cycle, or the like. Additionally or alternatively, thethreshold for one pixel may be different than for another pixel. Forexample, the threshold may be adjusted based on an expected amplitude ofthe received reflection(s), a level of ambient light received on thepixel during (or immediately before and/or after) the scan cycle, or thelike.

FIG. 17E is a diagram illustrating an exemplary vehicle 110 with a LIDARsystem using measured properties across scan cycles to detect objects.For example, vehicle 110 may include the system of FIG. 13, describedabove. Vehicle 110 may include at least one pixel associated withregions of a field of view. Moreover, vehicle 110 may scan theassociated regions over the course of multiple scan cycles, e.g.,scanning region 1701 in the first scan cycle and scanning regions 1701′in the second scan cycle. Vehicle 110 may receive signals associatedwith objects, road markings, or the like during scan cycles, e.g.,object 1703.

In the example of FIG. 17E, a surface angle measured in region 1701 maybe aggregated with a surface angle measured in region 1701′. Thisaggregation may allow for more accurate classification of object 1703.Additionally or alternatively, the surface angle measured in region 1701and the surface angle measurement in region 1701′ may be used toincrease an object-existence-certainty level (or other confidence level)associated with signals from region 1701 and signals from region 1701′(e.g., after being aggregated). For example, the LIDAR system maydetermine that the surface angles are consistent with the existence of avehicle (i.e., object 1703) and thus increase theobject-existence-certainty level.

Although FIG. 17E uses surface angles, any measured property may beaggregated to improve classification and/or used to increase confidencescores of aggregated signals. For example, reflectivity levels, objectsurface compositions, or the like may be used similar to the surfaceangles described above.

FIG. 17F is a diagram illustrating an exemplary vehicle 110 with a LIDARsystem using detected motion across scan cycles to detect objects. Forexample, vehicle 110 may include the system of FIG. 13, described above.Vehicle 110 may include at least one pixel associated with regions of afield of view. Moreover, vehicle 110 may scan the associated regionsover the course of multiple scan cycles, e.g., a first scan of region1705 in a first cycle (shown on the left) and a second scan of region1705 in a second cycle (shown on the right). Vehicle 110 may receivesignals associated with objects, road markings, or the like during scancycles, e.g., object 1707.

In the example of FIG. 17F, object 1707 may undergo motion between scancycles. Accordingly, the LIDAR systems may determine one or moreproperties of the motion based on signals from the first cycle andsignals from the second cycle. For example, the LIDAR system maydetermine differences between representations of object 1707 in thefirst cycle and the second cycle, determine offset between locations ofobject 1707 (or of an identifiable part thereof) in the tworepresentations, determine one or more translation parameters of object1707 (or of an identifiable part thereof) in the two representations(e.g., rotation angles, expansion, or the like), and determine avelocity of object 1707. In some embodiments, the LIDAR system may usethe offset and/or the one or more translation parameters withoutproceeding to determine the velocity.

In any of the examples above, the motion information may be used toassist with classification of object 1707. For example, certain motionprofiles may be associated with vehicles and other motion profilesassociated with pedestrians, allowing for motion information to assistwith classification of an object as a vehicle or a pedestrian.Additionally or alternatively, the motion information may be used toincrease an object-existence-certainty level (or other confidence level)associated with signals from the first cycle and signals from the secondcycle (e.g., after being aggregated). For example, the LIDAR system maydetermine that the motion information is consistent with the existenceof a vehicle (i.e., object 1707) and thus increase theobject-existence-certainty level.

Although depicted with vehicle 110 being stationary, vehicle 110 mayalso undergo motion between scan cycles. Accordingly, the detectedmotion of object 1707 may be adjusted using motion information from oneor more sensors of vehicle 110 in order to determine relative motionbetween vehicle 110 and object 1707.

Although depicted separately, aggregation of one or more properties asdepicted in FIG. 17E and use of motion information depicted in FIG. 17Fmay be combined. For example, the LIDAR system may use both surfaceangles and/or other measured properties as well as motion information toincrease an object-existence-certainty level (or other confidence level)associated with signals aggregated across time. Additionally oralternatively, the LIDAR system may use surface angles and/or othermeasured properties as well as motion information to improve theaccuracy of a classification of the detected object.

FIG. 18A is a flowchart of method 1800 for using LIDAR system todetecting objects in a field of view. For example, method 1800 may beimplemented by at least one processor of a LIDAR system (e.g., at leastone processor 118 of LIDAR system 100) and/or by at least one processorwithin a body of a vehicle (e.g., processor 408 of housing 200B ofvehicle 110).

At step 1801, the at least one processor may control at least one LIDARlight source in a manner enabling light projected from the at least onelight source to vary over a plurality of scans of a field of view, thefield of view including a foreground area and a background area. Forexample, step 1801 may be performed in a manner similar to thatdescribed above for step 1501 of method 1500.

At step 1803, the at least one processor may receive from a group ofdetectors a plurality of input signals indicative of reflections of theprojected light from the field of view. For example, step 1803 may beperformed in a manner similar to that described above for step 1503 ofmethod 1500. Moreover, as explained above with respect to method 1500, arepresentation of a portion of the field of view associated with aplurality of pixels is constructible from the plurality of inputsignals.

At step 1805, the at least one processor may detect a possible existenceof an object in the background area based on first input signalsassociated with a first scanning cycle. For example, the at least oneprocessor may determine that the first input signals caused by one ormore reflections from the field of view represent possible reflectionsfrom an object. An object-existence-certainty level in the firstscanning cycle may be below a threshold.

As explained above with respect to FIGS. 17A-17D, the threshold may bedynamic. For example, the threshold may depend on an amplitude of theprojected light during the scan cycle, on a level of ambient lightreceived during (or immediately before and/or after) the scan cycle, orthe like. Additionally or alternatively, the threshold may depend on atype of detected object. For example, the threshold may be higher forobjects expected to have greater reflectivity (such as bright objects,vehicles or parts thereof, etc.) and lower for objects expected to havelesser reflectivity (such as pavement, pedestrians, etc.). Additionallyor alternatively, the threshold may depend on a distance to a detectedobject, e.g., from the at least one light source. For example, thethreshold may be adjusted downward when the time(s)-of-flight for thereflection(s) causing the input signal(s) is longer and adjusted upwardwhen the time(s)-of-flight for the reflection(s) causing the inputsignal(s) is shorter. Any of the adjustments discussed above may besubject to a minimum (such that the at least one processor may not lowerthe threshold below the minimum) and/or subject to a maximum (such thatthe at least one processor may not increase the threshold above themaximum).

In some embodiments, the threshold may comprise a range. For example,the threshold may be a static range such that a signal is only above thethreshold if exceeding the maximum of the range and is only below thethreshold if below the minimum of the range. Alternatively, thethreshold may be a dynamic range. For example, the threshold may be astatic range that is adjusted across pixels and/or adjusted across scancycles. Similar to the single threshold described above, any adjustmentsto the range may be subject to an absolute minimum (such that the atleast one processor may not lower the minimum of the range below theabsolute minimum) and/or subject to an absolute maximum (such that theat least one processor may not increase the maximum of the range abovethe absolute maximum).

In some embodiments, the at least one processor may adjust one or moreoperating parameters of the LIDAR system based on the input signals. Forexample, as depicted in FIG. 17A, when the object-existence-certaintylevel in the first scanning cycle is below the threshold, the at leastone processor may cause more light to be projected towards the objectduring the second scanning cycle compared to light was projected towardsthe object in the first scanning cycle. Additionally or alternatively,as depicted in FIG. 17A, when the object-existence-certainty level inthe first scanning cycle is below the threshold, the at least oneprocessor may increase a sensitivity level of at least some of the groupof detectors in the second scanning cycle compared to the sensitivitylevel of the at least some of the group of detectors in the firstscanning cycle.

In any of the embodiments described above, as depicted in FIG. 17B, asecond threshold may be used. This may ensure that only suspectdetections, e.g., an object-existence-certainty level having a mediumconfidence level (e.g., a confidence level which exceeds a requiredlower threshold), are aggregated while noise remains filtered out.

At step 1807, the at least one processor may detect a possible existenceof the object based on second input signals associated with a secondscanning cycle. For example, the at least one processor may determinethat the second input signals caused by one or more reflections from thefield of view represent possible reflections from an object. Anobject-existence-certainty level in the second scanning cycle may bebelow the threshold.

In some embodiments, the at least one processor may select the secondinput signals based on a direction associated with the first inputsignals and at least one kinetic parameter of an object in the FOV. Forexample, as depicted in FIG. 17F, motion information associated with thedetected object may be used determine which second input signals to use.Accordingly, the at least one processor may ensure that the second inputsignals are associated with the same (or at least partially overlapping)area of the object as the first input signals. Additionally oralternatively, the at least one processor may use motion informationfrom one or more sensors of the LIDAR system to compensate for motion ofthe LIDAR system.

Accordingly, as explained above, the first and second input signals maybe associated with a same portion of the field of view or with differingbut overlapping portions of the field of view. Moreover, in any of theembodiments discussed above, the second scanning cycle may beconsecutive with the first scanning cycle or non-consecutive with thefirst scanning cycle. For example, in a cascading aggregation, one ormore scanning cycles may be skipped if input signals from that scanningcycle are below a second threshold (and therefore classified as noiserather than suspect detections).

At step 1809, the at least one processor may aggregate the first inputsignals associated with the first scanning cycle and the second inputsignals associated with the second scanning cycle to detect an existenceof the object at an object-existence-certainty level higher than thethreshold. As explained above with reference to FIGS. 17E and 17F, theat least one processor may take into account an angular orientation of asurface identified in the first input signal and second input signal, areflectivity level identified in the first and second input signals, orthe like to increase the object-existence-certainty level. In addition,the at least one processor may take into account a velocity of a vehiclehaving the LIDAR system when combining the first input signals and thesecond input signals. Additionally or alternatively, the at least oneprocessor may detect a change of at least one of a presence indication,a surface angle, and a reflectivity level in the second input signalscompared to the first input signals, and take into account the detectedchange to increase the object-existence-certainty level.

Step 1809 may further include initially processing individually inputsignals associated with the first and second scanning cycles. When theobject-existence-certainty level in both the first and second scanningcycles is below the threshold but higher than a second threshold, the atleast one processor may combine input signals associated with the firstand second scanning cycles. Similar to the threshold discussed above,the first threshold and/or the second threshold may be static.Alternatively, the first threshold and/or the second threshold may bedynamic. In either embodiment, the first threshold and/or the secondthreshold may comprise a range.

Additionally or alternatively, method 1800 may include aggregating inputsignals from a third scanning cycle when the object-existence-certaintylevel associated with the combined the first and second input signals islower than the threshold to detect the existence of the object at theobject-existence-certainty level higher than the threshold. For example,as described above with respect to FIG. 17D, a cascading aggregation maybe implemented.

Additionally or alternatively, method 1800 may include taking intoaccount input signals from a third scanning cycle when theobject-existence-certainty level associated with the first and secondinput signals is above the threshold, in order to obtain a pointresolution for the object greater than a point resolution achievablewith the first and second input signals alone. For example, as describedabove with respect to FIG. 17E, a cascading aggregation may beimplemented.

In any of the embodiments listed above, method 1800 may includeaccessing stored classification information, and based on theclassification information and the first and second input signals,classify the object. For example, the stored information may comprisestored signals associated with classifications such that an aggregationof the first and second input signals may be mapped to one or morestored signals, producing possible classifications. Any possibleclassifications may have associated confidence levels, e.g., based onthe level of match between a stored signal and the aggregation.Moreover, as described above with respect to FIGS. 17E and 17F, measuredproperties and/or motion information may further supplement the firstand second input signals to improve classification.

Although described using object-existence-certainty level, method 1800may be implemented using any other measurement (or confidence levelassociated with a measurement), such as reflectivity, surface angle,surface composition, or the like. Accordingly, method 1800 may allow forsignal aggregation to improve confidence regarding a property of anobject in addition to or in lieu of existence of the object itself.

Method 1800 combined with method 1500. For example, signals from aplurality of pixels may be aggregated using method 1500, and thisaggregated signal may further be aggregated with signals from anotherscan cycle using method 1800. The aggregation across time may beperformed first or the aggregation across space may be performed first.In some combinations, signals from a first scan cycle may be aggregatedfrom a plurality of pixels and then aggregated with signals from feweror greater pixels in a second scan cycle. Additionally or alternatively,signals from a first scan cycle may be aggregated from a first pluralityof pixels and then aggregated with signals from a second plurality ofpixels in a second scan cycle, the second plurality of pixels having atleast some pixels that are not in the first plurality of pixels.

FIG. 18B is a flowchart of method 1850 for using LIDAR system todetecting objects in a field of view. For example, method 1850 may beimplemented by at least one processor of a LIDAR system (e.g., at leastone processor 118 of LIDAR system 100) and/or by at least one processorwithin a body of a vehicle (e.g., processor 408 of housing 200B ofvehicle 110).

Steps 1851 and 1853 may be performed in manners similar to steps 1801and 1803, respectively, as described above.

At step 1855, the at least one processor may classify an object based onfirst input signals associated with a first scanning cycle. For example,the at least one processor may determine that the first input signalscaused by one or more reflections from the field of view match one ormore stored or otherwise known signals associated with one or moreclassifications. The one or more classifications may have associatedconfidence levels representing the certainty of the classification. Theconfidence level(s) in the first scanning cycle may be below athreshold. Accordingly, step 1855 may be performed in a similar mannerto step 1805 but using object classification rather than objectdetection.

At step 1857, the at least one processor may classify an object based onsecond input signals associated with a second scanning cycle. Forexample, the at least one processor may determine that the second inputsignals caused by one or more reflections from the field of view matchone or more stored or otherwise known signals associated with one ormore classifications. The one or more classifications may haveassociated confidence levels representing the certainty of theclassification. The confidence level(s) in the second scanning cycle maybe below a threshold. Accordingly, step 1857 may be performed in asimilar manner to step 1807 but using object classification rather thanobject detection.

At step 1859, the at least one processor may aggregate the first inputsignals associated with the first scanning cycle and the second inputsignals associated with the second scanning cycle to classify the objectwith a confidence level higher than the threshold. Accordingly, step1859 may be performed in a similar manner to step 1809 but using objectclassification rather than object detection.

Moreover, similar to method 1800, method 1850 may include taking intoaccount an angular orientation of a surface identified in the firstinput signal and second input signal, a reflectivity level identified inthe first and second input signals, or the like to increase theconfidence level of the classification. In addition, the at least oneprocessor may take into account a velocity of a vehicle having the LIDARsystem when combining the first input signals and the second inputsignals. Additionally or alternatively, the at least one processor maydetect a change of at least one of a presence indication, a surfaceangle, and a reflectivity level in the second input signals compared tothe first input signals, and take into account the detected change toincrease the confidence level of the classification.

Method 1850 combined with method 1500. For example, as explained abovewith respect to the combination of method 1800 and method 1500, signalsmay be aggregated across space and time in order to improveclassification in addition to or in lieu of improving detection.

Binning and Non-Binning of Sensor Pixel Outputs

LIDAR systems (such as the LIDAR systems described above) may bedeployed on any platform where distance ranging information to objectsin a platform environment may be useful. In some cases, for example, thedisclosed LIDAR systems may be deployed on a vehicle to provide ranginginformation relative to objects in an environment of the vehicle. Objectdetection capability may depend on a variety of factors, includingdistance between the object and the LIDAR system, reflectivity of theobject, ambient light levels, interfering light reflections, etc. Insome cases, objects in the foreground that are relatively close to theLIDAR system may be detected more easily as compared to objects that aremore distant from the LIDAR system. Also, objects exhibiting relativelylow reflectivity characteristics may be more difficult to detect ascompared to higher reflectivity objects. For example, there may beinstances when a LIDAR system may detect distant, highly reflectiveobjects more readily that close-in objects of low reflectivity (evenwhere the more distant objects may occupy fewer field of view pixels(e.g., a single FOV pixel), as compared to a closer object).

The disclosed LIDAR systems may enhance detection capability of objectswithin a LIDAR FOV. For example, the disclosed techniques of processingLIDAR sensor outputs, described in detail below, may enable and/orenhance detections of distant objects, low reflectivity objects, etc.Such techniques may aid in identifying objects that otherwise would havegone undetected and/or may increase confidence levels in rangeinformation, object envelope detection, etc. relative to detectedobjects.

In some embodiments, the challenges associated with object detection maybe addressed through techniques associated with processing of LIDARsensor outputs. Such techniques, for example, may include binningprocessing schemes and non-binning processing schemes. As will bediscussed in detail below, the disclosed LIDAR system may apply on thesame frame both a non-binned detection scheme (e.g., signals ofindividual sensor pixels may be analyzed individually) and a binneddetection scheme (e.g., signals of two or more sensor pixels may beconsidered together or otherwise combined).

Binning of sensor-element outputs may be performed in any suitablemanner and may enable detection of distant objects and/or detection ofobjects of lower reflectivity. For example, binning of sensor outputsmay be selectively applied based on object distance from the LIDARsystem and/or object reflectivity. In some cases, the disclosed LIDARsystem may process input signals (e.g., sensor output signals)associated with objects in the foreground individually on a sensor-pixelby sensor-pixel basis and may bin the input signals associated with anobject in the background. Such binning schemes may combine the outputsof a plurality of detector-elements, which in some cases, may increasedetection sensitivity or may increase a confidence level associated withobject detections.

Binning of LIDAR sensor outputs may be performed as part of a sensoroutput processing technique. For example, a processor may receive asinput signals outputs from individual LIDAR sensor-elements (e.g.,pixels associated with one or more LIDAR sensors). In some cases, thesensor pixel output signals may be directly provided to one or moreprocessors as input signals to the processor(s). In other cases, thesensor pixel output signals may be indirectly provided to one or moreprocessors as input signals to the processor(s). For example, the sensorpixel output signals may be conditioned, amplified, etc. before beingprovided as input signals to the processor(s). The individual sensorpixel outputs may be stored and used by a processor either individuallyor combined in various groupings. For example, two or more sensor pixeloutput signals provided as input signals to a processor may be summedaccording to a predetermined binning scheme or according to a binningscheme determined on the fly. Such binning schemes may provide orenhance various LIDAR functions (e.g., object detection under certainconditions, ranging, etc.).

Because the sensor outputs may be used or re-used individually or invarious different combinations, the disclosed binning techniques differfrom signal aggregration configurations. For example, as describedabove, in some cases, a LIDAR sensor may include a plurality of pixelsthat each may correspond to an individual SiPM or other type of lightsensitive element. In some cases, an individual SiPM may constitute aplurality of light sensitive subelements (e.g., SPADs). The outputs ofmultiple SPADS within a single SiPM “pixel” are often aggregatedtogether to provide the output of the corresponding SiPM (e.g., a sensorpixel). Such aggregation, however, is generally not performedselectively, as it is performed on the hardware level. Further, in suchhardware-based aggregation, the outputs of individual SPADs are not bepreserved after aggregation.

On the other hand, the disclosed binning techniques may selectivelyoccur as part of a processing phase where outputs of two or more sensorpixels (e.g., two or more SiPM outputs) may be combined or otherwiseconsidered together in performing certain LIDAR functions. Because thisprocess may be selective, certain SiPM or pixel outputs may beconsidered individually, may be selectively combined with certain otherSiPM outputs, may be uncombined from SiPM outputs, and/or may berecombined with the same or different SiPM outputs. Simple aggregationof sensor subelement outputs by combining multiple sensor subelementoutputs together without selectivity does not provide the samefunctionality as the disclosed binning techniques. For example, suchhardware-based aggregation techniques cannot provide an ability toconsider subelement outputs individually in some cases while consideringdifferent selected combinations of subelement outputs in other cases.

The disclosed binning techniques may be selectively performed by atleast one LIDAR processor and may be accomplished by performing one ormore logic operations or any other suitable combinatory technique on twoor more sensor pixel outputs. For example, binning of two or moresensor-pixel outputs may be based on or include signal addition, centerof gravity determinations, binary mapping, or others. For purposes ofthis disclosure, a sensor element (e.g., an individual SiPM) may be usedinterchangeably with the term sensor pixel.

Such a system may provide various potential benefits. For example, byutilizing binning and non-binning processing schemes by the samesystem—especially if applied on overlapping groups of detectionsignals—the system may offer increased detection capability undercertain conditions. Additionally, the use of binning and non-binningprocessing schemes may enhance confidence level in detections, providemore accurate range information, reflectivity data, angular and positioninformation, and/or may also increase efficiency. For example, thesystem may require low processing requirements with bins employed andmay unbin, as needed.

Moreover, by utilizing both binning and non-binning processing schemes,the system may avoid constraints associated with non-selective sensoroutput aggregation. In some prior art systems, outputs of two or moresensor-element are irreversibly combined together in a fixed manner. Insuch cases, there is no capability for selectively considering theunderlying sensor outputs individually, or to selectively sensor-elementconsider different combinations of sensor-element outputs. Rather, suchconstant sensor output aggregation architectures provide fixedcombinations of element outputs in all cases without selectivity (e.g.,a group of SPAD outputs are summed together to provide an SiPM output,and the processor has no way of accessing the individual SPAD signalsbefore their summation). In such cases, the underlying sensor-pixelsub-element (e.g., SPADs) outputs may be unavailable for individualprocessing. For example, individual SPAD outputs in an SiPM may beaggregated together without an ability to selectively consider any ofthe SPAD outputs individually.

On the other hand, the disclosed LIDAR system and binning techniques maypreserve individual sensor-element contributions, which may allow forselective binning, unbinning, and rebinning in different combinationsand configurations. In many cases, such binning, unbinning, and/orrebinning may be accomplished without the need to collect more sensordata. In some embodiments, the underlying individual contributions ofsensor-elements outputs may be preserved (e.g., by storage in memory)such that the individual contributions may be selectively andindividually processed/analyzed in some cases or processed/analyzedtogether with one or more other sensor-element outputs in other cases.As a processing technique, binning of sensor-element outputs may beselectively performed. Even after two or more sensor pixel outputs arebinned together, those same sensor pixel outputs may later be unbinnedand re-binned in different combinations (e.g., with other sensor pixeloutputs). For example, where a particular selected binning arrangementdoes not provide a desired performance level, detection sensitivity,etc., a different bin of sensor pixel outputs including any combinationof the same or different sensor outputs may be selectively binned andanalyzed. Such selective binning and unbinning may be performed relativeto sensor pixel outputs captured during a single FOV frame scan orcaptured across FOV frame scans.

In the presently disclosed embodiments, binning may be performedaccording to a predetermined pattern (e.g., certain pixels associatedwith near field regions of the LIDAR FOV may remain unbinned while otherpixels associated with far field regions of the LIDAR FOV may be binnedtogether during processing). On the other hand, in some cases, binning,unbinning, and/or rebinning in different combinations may be performeddynamically during operation of the LIDAR system (e.g., based ondetection related feedback, observed system performance, etc.).

The disclosed systems may have a variety of configurations. In oneexample, the disclosed systems may include LIDAR system components forillumination and collection of reflections (e.g. deflectors, actuators,light sources, and sensors), LIDAR system components for processing ofinput signals and outputting information indicative of object distance(e.g., one or more processors).

Any light source may be used depending on the requirements of aparticular application. In some cases, the light source may illuminate afield of view. In some cases, as depicted in the example of FIG. 1A,processor 118 of LIDAR system 100 may coordinate operation of lightsource 112 with the movement of light deflector 114 by actuator 302 ofFIG. 3A in order to illuminate a field of view. Of course, any lightsource configured to operation in any desired mode of operation (e.g.,continuous wave, pulsed, etc.) may be used. It should also be noted thatwhile references are made to system 100 shown in FIG. 1A and otherfigures, the disclosed binning techniques may be applicable to theoutput signals generated by light sensitive detectors other than thosedescribed herein. The disclosed binning techniques are also applicableto light sensitive detectors employed in various types of LIDAR systems(e.g., scanning, non-scanning (flash), etc.) or in other types ofranging, detection, imaging systems, etc.

Any deflector may be used depending on the requirements of a particularapplication. In some cases, processor 118 of LIDAR system 100 may beconfigured to control a light deflector in a scanning cycle to deflectlight from the light source such that during the scanning cycle thelight deflector moves through a plurality of different instantaneouspositions. In some embodiments, each instantaneous position of the lightdeflector may correspond to a different region of the LIDAR FOV (e.g.,region C4 in FIG. 19B).

Any one or more LIDAR sensors and/or LIDAR sensor-elements may be used,depending on the requirements of a particular application. As previouslynoted, a suitable LIDAR sensor may be comprised of an array of lightsensitive elements (e.g., SPADs) that are each sensitive to incidentphotons reflected from a objects in a region of the LIDAR field of view(e.g., region C4 in FIG. 19B). As shown in FIGS. 4A-4C, groups of SPADsmay be arranged together, where each grouping of SPADs may correspond toan SiPM, where each individual SiPM of the LIDAR sensor may provide apixel level output of the LIDAR sensor. In some cases, the SPADs oravalanche photo diodes that make up a LIDAR sensor may be formed on acommon silicon substrate. In one example, a distance between SPADs maybe between about 10 μm and about 50 μm, wherein each SPAD may have arecovery time of between about 20 ns and about 100 ns. As described withrespect to FIG. 4A, sensor 116 may include a plurality of detectionelements 402 for detecting photons of a photonic pulse reflected backfrom field of view 120. The detection elements may all be included indetector array 400, which may have a rectangular arrangement (e.g. asshown) or any other arrangement.

In an exemplary operation, scanning unit 104 of FIG. 2A may include areturn deflector 114B that directs photons (reflected light 206)reflected from an object 208 within field of view 120 toward sensor 116.The reflected light may be detected by individual SPADs within sensor116, and outputs of groupings of individual SPADs may be aggregatedtogether to provide individual SiPM or pixel outputs. Based on the pixeloutputs, processing unit 108 may determine information about an object(e.g., the distance to the object, etc.). In some cases, the outputsfrom a group of pixels may be used to form a LIDAR image of the LIDARFOV. For example, the outputs from pixels of sensor 116 may be used toimage area C4 of FIG. 19B along with other regions of the FOV.

As will be discussed in detail below, various binning operations may beperformed consistent with the presently disclosed embodiments. In somecases, binning of two or more sensor pixel outputs may be implementedaccording to a predetermined scheme (e.g., certain sensor pixel outputsmay be considered together, combined, etc. according to a predeterminedpattern of sensor pixels). In other cases, binning of sensor pixeloutputs may be accomplished based on observed characteristics of objectsor other aspects of a scene associated with the LIDAR FOV. For example,where no objects are detected or where objects are detected, but notwith a desired SNR (e.g., objects of low reflectivity or more distantobjects), certain sensor pixel outputs may be binned together to enhanceobject detection and LIDAR system output.

In some embodiments, the LIDAR system may include at least one processorconfigured to control at least one LIDAR light source for illuminating afield of view. During operation, as previously noted, the processor mayreceive from a group of light detectors of the LIDAR sensor a pluralityof input signals indicative of reflections of light from objects in thefield of view. Each light detector of the LIDAR sensor may correspond toa pixel of the sensor. In some cases, each light detector may include anSiPM, which includes a plurality of light sensitive subelements (e.g.,SPADs or other type of light sensitive element). In some embodiments, aspreviously described, a light deflector may be scanned through aplurality of different instantaneous positions. At each instantaneousposition, the LIDAR sensor may output a plurality of signalscorresponding to one or more of the pixel outputs of the LIDAR sensor.These pixel outputs may be provided to at least one LIDAR processor suchthat over the course of a full scan of the LIDAR FOV over a plurality ofinstantaneous positions of the light deflector, the processor mayreceive a plurality of input signals from the LIDAR sensor thatrepresent sensor pixel outputs collected at the instantaneous positionsof the deflector, where each instantaneous position of the deflector maycorrespond to a different field of view pixel of the LIDAR FOV. In someembodiments, sensor signals collected relative to a single field of viewpixel are combined together to provide a single data point in a pointcloud output of the LIDAR system. Thus, each data point in the pointcloud may be related to and derived from reflection signals receivedfrom a particular FOV pixel. The plurality of input signals collectedfrom the sensor may represent light reflection information collected bypixels of the sensor from different regions of the LIDAR FOV (e.g.,regions F4, E4, D4, C4, B4, A4, A3, . . . , D1, E1, F1 of FIG. 19B) asthe deflector is scanned over the LIDAR FOV. It should be noted thatwhile FIG. 19B shows a LIDAR FOV having 24 field of view pixels, a LIDARFOV may have fewer or more pixels. In some cases, the LIDAR FOV may havemany more pixels, as the LIDAR FOV may be divided into horizontal and/orvertical rows of pixels spaced apart by 0.2 degrees or any othersuitable angular spacing.

Any of the input signals received from the sensor may be binned togetheror considered individually, consistent with the presently disclosedembodiments. In some cases, input signals corresponding to sensor pixeloutputs all acquired from a particular region of the LIDAR FOV (e.g.,FOV pixel C4 in FIG. 19B) may be binned together. For example, FOV pixelC4 may overlap with a distant object or an object of low reflectivity.As a result, a light reflection received at the LIDAR sensor may resultin a non-zero output signal being generated by one or more of the sensorpixels. However, individually, the sensor pixel outputs may each fallbelow a detection threshold, may exhibit poor signal to noise ratios, orexhibit another characteristic rendering a positive detectiondetermination or an accurate range determination to be difficult orimpossible. In such cases, sensor pixel outputs may be binned together,and the sensor pixel outputs may be summed, for example. Binning sensorpixel outputs by summing, for example, may result in a combined outputthat exceeds a detection threshold, offers a better signal to noiseratio, enables more accurate ranging, etc.

In other cases, input signals corresponding to sensor pixel outputsacquired from different regions (e.g., FOV pixels) of a LIDAR FOV (e.g.,region C4 and region C3 in FIG. 19B) may be binned together. Binning ofinput signals received from any combination of regions or FOV pixels ofthe LIDAR FOV may also be performed. In some cases, sensor pixel outputsassociated with a first region of the LIDAR FOV may be processedaccording to a first binning scheme (e.g., unbinned, 2×2 bins, etc), andsensor pixel outputs associated with a second region of the LIDAR FOV,which may be spatially separated from the first region of the LIDAR FOV,may be processed according to a different binning scheme. In some cases,however, the first region of the LIDAR FOV and the second region of theLIDAR FOV processed according to different binning schemes may at leastpartially overlap.

In some cases, a same group of detector elements (e.g., a group ofsensor pixels) may generate both a first subset of input signalsprocessed according to a first binning scheme and a second subset ofinput signals processed according to a second binning scheme differentfrom the first. For example, the first subset of input signals may beacquired by the detector elements during a first portion of a scan of aLIDAR FOV (e.g., over a first period of time during capture of a firstframe of the LIDAR FOV), and the second subset of input signals may beacquired by the same detector elements during a second portion of aLIDAR FOV scan (e.g., during a second period of time during capture ofthe first frame of the LIDAR FOV later than the first period of time) orduring a subsequent scan of the LIDAR FOV altogether (e.g., during acapture of a second or later frame of the LIDAR FOV).

According to an exemplary binning technique, the processor may process afirst subset of the received input signals associated with a firstregion of the field of view (e.g., an FOV pixel in the FOV) to detect afirst object in the first region, wherein processing the first subset isperformed individually on each input signal of the first subset of theinput signals. In the example scene shown in FIG. 19B, the plurality ofinput signals collected from the collection of sensor pixels relative toregion D1 of the FOV (e.g., FOV pixel D1) may be processed by theprocessor without binning That is, the pixel outputs of the LIDAR sensorcollected relative to FOV pixel D1 may be processed individually. Inthis example, an object (e.g., pedestrian 1901) may be located close tothe LIDAR system, or in a near field region of the LIDAR system, andindividually processing sensor pixel outputs, without binning, may besuitable for detection, ranging, etc.

On the other hand, the processor may be configured to process a secondsubset of the input signals associated with a second region of the fieldof view to detect at least one second object in the second region,wherein the at least one second object is located at a greater distancefrom the at least one light source than the first object and whereinprocessing of the second subset includes processing together inputsignals of the second subset. For example, returning to FIG. 19B, thesecond subset of input signals may correspond to sensor pixel outputsacquired relative to region C4 (e.g., the FOV pixel corresponding toregion C4) of the LIDAR FOV. Region C4 may include, for example, avehicle 1903 that is located further from the LIDAR system thanpedestrian 1901. In such cases, whether processing occurs according to apredetermined binning scheme or a binning plan developed dynamically,sensor pixel output signals acquired relative to region C4 (e.g., as theLIDAR deflector is positioned during a scanning cycle at aninstantaneous position that corresponds to FOV pixel C4) may beprocessed by binning (e.g., combining, considering together, etc.) twoor more of the pixel output signals acquired relative to region C4 thatare provided as inputs to the processor. In such a process, the LIDARsystem may more readily detect vehicle 1903, may determine more accuratedistance measurements relative to vehicle 1903, may have a higherconfidence level in measurements made relative to vehicle 1903, etc.

Further, processor 118 may be configured to process a third subset ofthe input signals associated with a third region of the field of view todetect a third object in the third region. The third object may belocated at a greater distance from the light source than the at leastone second object. Additionally, processing of the third subset of inputsignals may include combining input signals of the third subset (e.g,binning those signals together). Based on the detected third object, theLIDAR system may output information including a distance to the thirdobject. Rather than, or in addition to, being more distant than thesecond object described above, the third object may be less reflectivethan the second object. In addition to providing distance informationrelative to any of the first, second, or third detected objects (or anyother detected objects), the LIDAR system may also provide additionalmeasurements including (but not limited to) any one or more of thefollowing measurements: a velocity of any of the detected objects, asurface angle of of any of the detected objects, a reflectivity level ofof any of the detected objects, ambient light associated with of any ofthe detected objects, and a confidence level in any of thesemeasurements (e.g., based on an SNR, etc.).

While the LIDAR FOV of FIG. 19B is diagrammatically represented in atwo-dimensional view, the regions of the LIDAR FOV (e.g., each FOVpixel) represents a three-dimensional section within the LIDAR FOV. Forexample, as shown in FIG. 19D, each FOV pixel to which LIDAR light isprojected and from which reflected LIDAR light is collected includes athree-dimensional volume in space that spans a range of distance valuesfrom the LIDAR system 100 (e.g., beginning a the LIDAR system lightoutput as the origin and extending over the entire operating range ofthe LIDAR (e.g., 200 m, 500 m, or beyond). As an example, a first FOVpixel 1920 in the LIDAR FOV may extend from LIDAR system 100 at adistance d0, through a distance d1 (a distance that may correspond withat least a portion of the fire hydrant), out to a distance d2 (adistance to a normal plane that may intersect at least a portion of thepedestrian), and beyond.

When the deflector is located in an instantaneous position correspondingto FOV pixel 1920, the sensor may receive LIDAR light reflected from aportion of the fire hydrant overlapping with FOV pixel 1920. In somecases, in response to the received LIDAR light reflected from the firehydrant (especially where the fire hydrant is a highly reflective objector where distance d1 is relatively small—10 m, 15 m, 20 m, etc.) thesignals generated by more than one sensor pixel may be sufficient toaccurately detect the presence of a portion of the fire hydrantoverlapping FOV pixel 1920 and/or determine a distance range to theoverlapping portion of the fire hydrant. In such cases, the sensor pixeloutputs may be analyzed without binning.

In another example, however, when the deflector is located in aninstantaneous position corresponding to FOV pixel 1922, the sensor mayreceive LIDAR light reflected from a portion of the pedestrianoverlapping with FOV pixel 1922. In some cases, in response to thereceived LIDAR light reflected from the pedestrian (especially where thereflectivity of the pedestrian is low or where distance d2 is relativelylarge—50 m, 75 m, 100 m, 150 m, etc.) the signals generated byindividual sensor pixels collecting the received reflected light may beinsufficient on their own to accurately detect the presence of theportion of the pedestrian overlapping FOV pixel 1922 and/or determine adistance range to the overlapping portion of the pedestrian. In suchcases, two or more of the sensor pixel outputs may be combined andanalyzed together by applying any of the binning schemes describedherein.

With reference to the example shown in FIG. 19D, based on thenon-binned, individual sensor pixel outputs acquired relative to FOVpixel 1920, the LIDAR processor may determine a distance to a portion ofthe fire hydrant and may output information indicative of a distance tothe fire hydrant (e.g., a data point in a point cloud representing adistance to a location on the fire hydrant (e.g., corresponding to acenter of FOV pixel 1920)). Similarly, based on at least some of thebinned sensor pixel outputs acquired relative to FOV pixel 1922, theLIDAR processor may determine a distance to a portion of the pedestrianand may output information indicative of a distance to the portion ofthe pedestrian overlapping with FOV pixel 1922.

Returning to FIG. 19B, in some cases, a first region of the field ofview (e.g., FOV pixel D1) may include relatively near objects and,therefore, may be associated with a foreground portion of the LIDARfield of view. A second region of the field of view (e.g., FOV pixel C4in FIG. 19B) may include more distant objects and, therefore, may beassociated with a background portion of the field of view. A foregroundportion may be associated with the near field region of the LIDAR system(e.g., between 5 and 50 meters, between 5 and 100 meters, between 1 and50 meters, between 1 and 100 meters, etc.). A background portion may beassociated with a mid or far field region relative to the LIDAR system(e.g., between 50 and 500 meters, between 100 and 500 meters, or more).In some embodiments, the foreground and background regions may bedetermined based on distance. For example, if the system may identify anobject at a certain distance, anything closer to the LIDAR system thanthe object may be considered foreground and anything farther from theLIDAR system than the object may be considered background. In someembodiments, foreground and background positions may be determined basedon results of the scan. In some embodiments, the foreground andbackground position may be determined based on map/topologicalsimultaneous localization and mapping (SLAM). In some embodiments, theforeground and background regions may be determined based on previousframes. In some embodiments, the foreground and background position maybe determined based on signal levels. In some embodiments, first andsecond subsets of the input signals are received during the scanningcycle.

Returning to FIG. 19A, a diagrammatic illustration is provided of ascene 1900 including an object, i.e., pedestrian 1901, in a foreground,near field region and a second object, i.e., vehicle 1903, inbackground, far field region of the LIDAR FOV (e.g., FOV 120). As noted,FIG. 19B provides a diagrammatic illustration of a scene 1910 thatcorresponds to the scene 1900 of FIG. 19A. FIG. 19B, however, identifiesexemplary regions of the LIDAR FOV (e.g., FOV pixels A1, B1 . . . E4,F4, etc.). Each of the regions of the LIDAR FOV may correspond to aninstantaneous position of the LIDAR deflector through which the LIDARdeflector passes during a single scan of the LIDAR FOV. In someembodiments, the LIDAR system sensor may receive more reflected lightfrom pedestrian 1901 in the near field region than from vehicle 1903 inthe far field region. In some cases, the difference in reflected lightmay be due to the difference in distance. In other cases, the differencein reflected light may be associated with the reflectivity of theobject. As noted above, while only 24 FOV pixels are shown in FIG. 19B,there may be many more FOV pixels in practice, as an FOV pixel may occurevery 0.2 degrees in horizontal and/or vertical directions of the FOV.

In some embodiments, sensor 116 of FIG. 1A (or any other LIDAR sensor)may enable an association between the reflected light from a singleportion of field of view 120 (e.g., a single FOV pixel) and the outputsof individual light detectors (e.g., sensor pixels each constituting anSiPM). As the LIDAR deflector including scans through a plurality ofdifferent instantaneous positions, light reflected from each particularLIDAR FOV region (e.g., FOV pixel) corresponding to the differentinstantaneous positions of the deflector may be acquired. The incidentreflected light may be sensed by detection elements 402 (e.g., SPADs),and outputs of groups of SPADs may provide the output of a single pixel404 (e.g., an SiPM), as represented by FIG. 4A. As represented by FIG.4C, LIDAR sensor 116 may include a plurality of detectors (e.g., 410Aand 410B) corresponding to sensor pixels, where each pixel may include aplurality of light sensitive elements (e.g., detection elements 402(e.g., SPADs)).

Returning to the example of FIG. 19B, as a result of scanning thedeflector through a plurality of instantaneous positions, sensor 116 mayacquire reflected light at different times (e.g., sequentially) fromeach of the scanned regions of the LIDAR FOV (e.g., regions F4, E4 . . .E1, F1). Thus, when sensor 116 collects reflected light from region D1,many of the pixels associated with sensor 116 may receive reflectedlight from pedestrian 1901 (in the example where a sensor having morethan one pixel sensor is used to scan one FOV pixel at a time).Similarly, when sensor 116 collects reflected light from region C4, manyof the pixels associated with sensor 116 may correspond to vehicle 1903.Because vehicle 1903 is in a far field region of the LIDAR system, notall pixels of sensor 116 that overlap with vehicle 1903 when imagingregion C4 may provide output indicative of the receipt of reflectedlight. In some cases, this may be because less reflected light isreceived at the sensor 116 from more distant objects. A similar effectmay exist as objects exhibit less reflectivity. For at least thisreason, outputs of pixels associated with sensor 116 may be selectivelybinned during processing by the LIDAR processor.

Looking more specifically to region C4 of the LIDAR FOV, as shown inFIG. 19B, when the deflector is positioned such that reflected LIDARlight is acquired from FOV pixel C4, at least some of the sensor pixelsof sensor 116 may output a signal indicative of light received from FOVpixel C4 of the LIDAR FOV. Each pixel of sensor 116 may generate anoutput, and these outputs of the sensor pixels may be provided to theLIDAR processor as input signals. Each pixel output may be indicative ofreflected light received by the light sensitive elements associated witha particular pixel.

The amount of reflected light received from vehicle 1903 may determine awhether a positive detection of vehicle 1903 can be made and/or whetheran accurate distance to vehicle 1903 may be determined. The amount ofreflected light received from vehicle 1903, however, depends on thedistance of vehicle 1903 from the LIDAR system and on the reflectivityof vehicle 1903. The farther vehicle 1903 is located relative to theLIDAR system and/or the less reflective that vehicle 1903 is, the lessreflected light may be received at the LIDAR system and detected bypixels of the LIDAR sensor. As a result, the output from any one sensorpixel may fall below a positive detection threshold or may exhibit apoor signal to noise ratio.

In such cases, the output of the pixels of the LIDAR sensor may notenable the LIDAR processor to accurately detect vehicle 1903, determinea distance to vehicle 1903, etc. A diagrammatic representation of suchan example situation is shown in FIG. 19C. As shown in FIG. 19C (notnecessarily to scale), a LIDAR sensor including multiple sensor pixelsrepresented by the grid overlaying region C4 of the LIDAR FOV maycollect reflected light from region C4 of the LIDAR FOV. In thisconceptual example, the sensor includes 64 pixels in an 8×8 array(including elements C-1, 4-1; C-2, 4-2 . . . C-8, 4-8). In the examplerepresented by FIG. 19C, only some of the sensor pixels generate anoutput signal indicative of received reflected light. As shown by sensorpixels in black in FIG. 19C, sensor pixels C-4, 4-3; C-7, 4-3; C-6, 4-4;among several others have generated output signals indicative ofreceived reflected light

On the other hand, several of the sensor pixels (e.g., C-3, 4-4; C-7,4-4; C-6, 4-6; etc.) have not produced output signals indicative ofreceived reflected light (or at least from which at least one signalfeature supercedes a detection threshold or differentiates from signalnoise to a sufficient degree to positively indicate received light). Insuch cases, binning of the sensor pixel outputs may increase detectionconfidence, signal to noise ratios, and/or accuracy of a distancedetermination relative to vehicle 1903.

In the example described above relative to FIGS. 19A-19C, the LIDAR FOVis scanned one LIDAR FOV pixel at a time using a LIDAR sensor havingmultiple sensor pixels. As as a result, each FOV pixel is associatedwith outputs from multiple sensor pixels. In such an example, thedescribed binning schemes may be used to combine one or more sensorpixel outputs relative to a single FOV pixel.

The disclosed binning techniques, however, may be applied to many otherexamples. In some cases, each single sensor pixel may be associated witha corresponding single FOV pixel. For example, in some cases, a LIDARsensor may include only a single pixel sensor, and the FOV may bescanned one FOV pixel at a time. In this case, a sensor pixel output maybe acquired for each different instantaneous position of the deflectorand in response to sensed reflected light collected at each differentinstantaneous position of the deflector. As a result, each acquiredsensor pixel output may correspond to a different FOV pixel. Forexample, as shown in FIG. 20A curve 2002 represents output signalsgenerated over time by the single pixel of a LIDAR sensor. As shown, thesensor pixel output may be acquired at each of three differentinstantaneous positions of the LIDAR deflector based on light projectedto the FOV pixels corresponding to those instantaneous positions of thedeflector. As a result, an acquired pixel output including signal 2004may be associated with FOV pixel 1, a sensor pixel output including 2006may be associated with FOV pixel 2, and a sensor pixel output includingsignal 2008 may be associated with FOV pixel 3. As shown, signal 2004associated with FOV pixel 1 may exceed a positive detection threshold2010. On the other hand, signals 2006 and 2008 associated with FOVpixels 2 and 3, however, may fall below the positive detection threshold2010. In such cases, it may be desirable to bin two or more of thesensor pixel outputs. For example, the sensor pixel outputscorresponding to FOV pixel 2 and FOV pixel 3 may be binned together by,e.g., summing together signals 2006 and 2008. Curve 2012 representssummed signals 2006 and 2008, which may exceed positive detectionthreshold 2010, provide a higher SNR, etc. Thus, summing the sensorpixel output signals corresponding to FOV pixel 2 and FOV pixel 3 mayprovide a higher detection confidence level and/or may enable a moreaccurate distance determination based on the summed/binned signals.

A LIDAR system may include as an output a point cloud where each datapoint represents a distance determination (e.g., to an object or portionof an object) made relative to a particular FOV pixel. Where signals2004, 2006, and 2008 of FIG. 20A are not binned, these signals (ifpossible) may be used to generate a distance determination relative toFOV pixel 1, FOV pixel 2, and FOV pixel 3, respectively. Spatially, thecenters of each of these FOV pixels will be separated from the centersof adjacent FOV pixels by an angular displacement determined by theangular change of the LIDAR deflector between sequential instantaneouspositions. Thus, where no binning is performed, the LIDAR point cloudassociated with the example in FIG. 20A will include three points,equally spaced angularly, each representative of a distance to adetected object or portion of an object in the three FOV pixels.

Where binning occurs, however, the points in the point cloud may not beequally spaced from one another. For example, if signals 2006 and 2008are summed as part of a binning process, a distance determinationrelative to the combined FOV pixel 2+3 may be based on summed signal2012. This distance determination may be represented by a data point inthe point cloud that is assigned to the center of the FOV regioncorresponding to combined FOV pixel 2 and FOV pixel 3. Spatially, thedata point assigned to the center of the combined area of FOV pixel 2and FOV pixel 3 will be farther from the data point representing FOVpixel 1 than a data point assigned to FOV pixel 2 in the absence ofbinning.

Of course, other binning schemes may be applied to the example of FIG.20A. For example, rather than summing the sensor pixel signalsassociated with FOV pixel 2 (2006) and FOV pixel 3 (2008), the sensorpixel signal associated with FOV pixel 1 (2004) could be binned/summedwith signal 2006 and/or 2008.

Additionally, because the signal information obtained over time fromeach sensor pixel relative to each FOV pixel may be stored in memory,binning may be performed relative to any two pre-acquired sensor pixeloutput signals. For example, a sensor pixel output signal associatedwith a first FOV pixel may be binned/summed with any other sensor pixeloutput signal acquired relative to any other FOV pixel, whether thesignals are acquired during a single FOV scan or during separate FOVscans. Such a binning capability may allow for combining sensor signaloutputs associated with adjacent FOV pixels along a horizontal scan line(e.g., as may be the case in FIG. 20A if signals associated with FOVpixel are binned with signals associated with FOV pixel 2, or signalsassociated with FOV pixel 2 are combined with signals associated withFOV pixel 3). The disclosed binning techniques, however, may also allowfor binning of sensor pixel output signals associated with verticallyadjacent FOV pixels (e.g., vertically adjacent FOV pixels located indifferent horizontal scan lines). Further still, sensor pixel outputsignals acquired with respect to any FOV pixel may be binned or combinedwith the sensor pixel output signals acquired with respect to any otherFOV pixel or group of FOV pixels, whether those FOV pixels are adjacentor contiguous or not.

Additionally, rather than binning sensor pixel output signals associatedwith different FOV pixels (e.g., a spatial binning of signals), binningmay also be performed relative to sensor pixel output signals acquiredat different times for a single FOV pixel. In one example, the binningof sensor pixel output signals acquired relative to a particular FOVpixel may be performed relative to signals acquired during one FOV framescan. For example, multiple light pulses can be provided to a single FOVpixel. Each projected light pulse may result in a corresponding receivedreflected light pulse at the sensor. In turn, each received reflectedlight pulse may result in a discrete, observable sensor pixel outputsignal spaced apart by a certain time period (which may correspond tothe time duration between light projections to the FOV pixel). Two ormore of these discrete sensor pixel output signals acquired at differenttimes for a particular FOV pixel during a single FOV frame scan may bebinned together. Because sensor pixel output signal information in thedisclosed embodiments may be stored in memory, in another example, thebinning of sensor pixel output signals acquired relative to a particularFOV pixel may also be performed relative to signals acquired duringdifferent FOV frame scans.

In another example, light may be projected to multiple FOV pixelssimultaneously. In such embodiments, a LIDAR sensor may be configured toinclude a plurality of sensor pixels, where together with appropriateoptical components, as needed, each sensor pixel may collect light froma different, corresponding FOV pixel. In one example, a scanning LIDARsystem may project a beam of light having a first dimension (e.g., aheight) equal to a single FOV sensor dimension and a second dimension(e.g., a width) that spans multiple FOV pixels. For example, a projectedbeam may simultaneously illuminate two, four, six or more FOV pixels ina single horizontal FOV scan line, and in such cases, the LIDAR sensormay have two, four, six or more sensor pixels configured to collectlight from corresponding FOV pixels.

In other example, such as a flash LIDAR system, light may besimultaneously projected to an entire FOV, and a LIDAR sensor having aplurality of sensor pixels may collect light reflected from the FOV. Insuch cases, the resolution of the LIDAR system corresponds with thenumber of pixels in the sensor. Further, in this example, the FOV pixelsare effectively defined by and correspond to the different sensorpixels.

In these examples, binning may be applied by the LIDAR processoraccording to any of the techniques described herein. For example, anysignals generated by any single sensor pixel in response to reflectedlight received from a particular FOV pixel may bebinned/summed/combined, etc. with any other signals generated by anyother single sensor pixel or group of sensor pixels. FIG. 20Bdiagrammatically illustrates one example of binning in a case wheresignals from four sensor pixels are collected simultaneously in responseto simultaneous illumination of four corresponding FOV pixels. In thisexample, a first curve 2050 represents the signal output from a firstsensor pixel, a second curve 2052 represents the signal output from asecond sensor pixel, a third curve 2054 represents the signal outputfrom a third sensor pixel, and a fourth curve 2056 represents the signaloutput from a fourth sensor pixel. At a first time, four FOV pixels maybe illuminated, and sensor pixels 1, 2, 3, and 4 may collect lightreflected from the corresponding four FOV pixels. In response, sensorpixels 1, 2, 3, and 4 may generate output signals 2060, 2062, 2064, and2066. Any combination of these signals may be binned together. Becauseeach of these signals exceeds a detection threshold 2080, binning may beless important. At a later time, for example, after the LIDAR deflectormoves to a new instantaneous location, four new FOV pixels may besimultaneously illuminated and sensor pixels 1, 2, 3, and 4 may collectreflected light from the corresponding four FOV pixels. In response,sensor pixels 1, 2, 3, and 4 may generate output signals 2070, 2072,2074, and 2076. In some cases these output signals may not exceeddetection threshold 2080. Any combination of signals 2070, 2072, 2074,and 2076 may be binned together and analyzed. For example, binning ofthe outputs of sensor pixels 1 and 2 may result in a curve 2082including summed signal 2090. Binning of the outputs of sensor pixels 3and 4 may result in a curve 2084 including summed signal 2092, andbinning of the outputs of sensor pixels 1, 2, 3, and 4 may result in acurve 2086 including summed signal 2094. Notably, each of summed signals2090, 2092, and 2094 may exceeds detection threshold 2080.

In addition to the binning combinations shown in FIG. 20B, any firstsensor pixel output signal, acquired at any time and for any FOV pixel,may be binned together with the output signals (acquired at any time)from any other sensor pixel or group of sensor pixels relative to anyFOV pixel or group of FOV pixels. In one example, signal 2060 could bebinned with signal 2070 (signals relative to different FOV pixels, butgenerated from the same sensor pixel 1); signal 2066 may be binned withsignal 2070 (signals from different sensor pixels, but from potentiallyadjacent FOV pixels); and any other combination of sensor pixel outputsignals may be binned together.

The LIDAR processor may apply any of various binning schemes to sensorpixel outputs (or to values associated with the received pixel outputs)acquired from the LIDAR sensor. For example, pixel outputs received bythe processor may be binned together in groupings such that at least onegroup of binned sensor pixel outputs includes two or more pixel outputssummed together. The groupings may have different sizes either relativeto an arrangement of sensor pixels (e.g., FIG. 19C embodiment) orrelative to an arrangement of FOV pixels (e.g., FIG. 20B embodiment).The groupings may be 1×2, 2×1, 2×2, 3×1, 2×3, 3×3, 4×4, etc. Anycombination of pixel outputs may be used. The binned groupings may becontiguous (e.g. where two or more of the grouped pixels touch oneanother) or discontiguous (e.g., where grouped pixels are spaced apartfrom one another). Further, not all binned pixel groupings need to havethe same size. Some binned groupings may have a size of 2×2 pixels,while others have a 4×4 or 8×4 grouping size, for example. Furtherstill, not all pixels of the sensor need be binned relative to aparticular region of the LIDAR FOV. For example, in some cases, onlycertain sensor pixel outputs relative to certain FOV pixels, whileothers are left unbinned. Binned sensor pixel groupings may bepredetermined (e.g., based on regions of an FOV associated with nearfield or far field objects) or may be determined on the fly (e.g., basedon feedback indicating received reflected light levels, noise levels,potential object detections, etc.).

In some cases, an applied binning scheme may depend on or may correlatewith amounts of light projected by the LIDAR to particular regions ofthe LIDAR FOV. For example, in some cases, certain regions of the LIDARFOV may be allocated with fewer light pulses from the LIDAR lightsource, as compared to other regions of the FOV Where less light issupplied to a certain region of an FOV, however, the likelihood ofdetecting objects in that region or accurately determining distancerelative to those objects, etc. may be reduced. To increase thelikelihood of making such detections, increase the accuracy of distancemeasurements, etc., any of the binning techniques described herein maybe employed based on an amount of light supplied to regions of the LIDARFOV. In one example, certain regions of the LIDAR FOV may be suppliedwith one light pulse from the LIDAR light source. In those regions, thesensor pixel outputs corresponding to two or more FOV pixels may bebinned according to any suitable binning scheme. In other regions, morelight pulses may be projected toward those FOV regions. In thoseregions, the sensor pixel outputs may remained unbinned. Or, in someembodiments, the sensor pixel outputs may be binned, but the binnedgroupings may have shapes or sizes different from the binned groupingsassociated with those FOV regions supplied with less projected light.

Processing of the binned pixel outputs may also be performed accordingto any suitable algorithm or technique. In some cases, the outputs ofbinned pixels may be summed together, and the summed value may beassigned to the binned pixel grouping as a whole (as noted relative toFIG. 20A).

In some embodiments, binning, the decision to implement a binningscheme, non-binning scheme, or more than one binning scheme on a part ofthe field of view may depend on the results of a non-binned detectionand/or a binning detection. The size or shape of bins and/or theapplication of pixel bins to certain FOV regions may be determineddynamically based on acquired detection information. For example, basedon the detection results relative to one or more regions of a LIDAR FOVduring a first scan, the LIDAR processor may apply a binning scheme ormay change an applied binning scheme to one or more regions of the FOVduring a current scan of the FOV or during a subsequent scan of the FOV.

Different binning schemes may also be applied relative to a single setof pixel outputs. For example, the LIDAR processor may be programmed toapply a first binning scheme to a set of pixel outputs (e.g., based onany of the factors described herein). Based on the detection resultsassociated with the first binning scheme, the processor may determinethat a different binning scheme should be applied to the set of pixeloutputs. In some cases, the different binning scheme may include more orfewer bins, differently shaped bins, larger or smaller bins, contiguousor non-contiguous bins, etc. In this way, a set of pixel outputs may beprocessed and re-processed using different binning schemes until adesired outcome is achieved.

In one example, as a result of a medium-low confidence of detection of afar field object from a non-binning scheme, processor 118 may implementa new binning scheme and re-process the outputs of the previouslynon-binned pixels. Additionally, for example, as a result of a lowconfidence of detection of a far field object determined with a binningscheme, processor 118 may alter the binning scheme (e.g., change thesize, shape, number of the bins, etc.). In another embodiment, processor118 may alter the binning scheme in order to improve confidence on amid-level object detection. In another embodiment, binning may depend onregions of interest. For example, where 20 meters ahead of the LIDARsystem is a region of interest, processor 118 may implement a binningscheme for non-binning sensor pixel output signals corresponding toreflections from objects up to 20 meters and binning sensor pixel outputsignals corresponding to reflections from objects from 20 meters and on.In another embodiment, binning may depend on results of previous frames.For example, processor 118 may detect an object ahead. As a result ofthe detection in the previous frame, processor 118 may adapt the binningscheme for greater confidence of detection or accuracy in distancedeterminations, etc., during subsequent scans. In another embodiment,binning may depend on computation and timing constraints. In anotherembodiment, binning may depend on host instructions. In anotherembodiment, binning may depend on optical budget allocation.

In some embodiments, as noted, processor 118 may individually process afirst subset of input signals provided by sensor pixels outputsassociated with a first region of the LIDAR FOV. For example, in theillustration shown in FIG. 19C, processor 118 may employ a non-binningapproach in analyzing the outputs of pixels acquired based reflectedlight collection from region D1 of the LIDAR FOV, which includespedestrian 1901 in a near field region of the field of view. As noted,processor 118 may also individually process signals 2060, 2062, 2064,and 2066 (FIG. 20B) without binning. In some cases, non-binnedprocessing may occur relative to objects in a near field region, andbinned processing may be performed relative to more distant and/or lessreflective objects. In some embodiments, processor 118 may implement anon-binning scheme in situations where the sensor pixel output signalsindicate that enough reflected light was individually received from eachcorresponding FOV pixel to provide a certain threshold confidence levelin detection (e.g., at least 50% certain, 75% certain, 95% certain, orany other percentage certainty of a detection). In some embodiments, thesystem may require a higher confidence level for detection of objects inthe near field region than for detection of objects in the far fieldregion. As a result, in some cases, binning may be applied relative tocertain pixel outputs associated with near field regions even wheresimilar outputs from far field regions would not warrant application ofa binning scheme.

In some embodiments, processor 118 may process a second subset of theinput signals provided by the LIDAR sensor pixels and may apply abinning scheme during processing of the second subset of input signals.For example, as discussed above, processor 118 may apply a binningscheme to pixel outputs associated with far field objects. The appliedbinning scheme may enable detection of an object that is located agreater distance from the light source than another object or that isless reflective than another object. In one embodiment, processor 118may consider sensor pixel output signals associated with objects in theforeground individually (e.g., without binning) and may bin sensor pixeloutput signals associated with an object in the background.

In some embodiments, some objects may be detected in the binning scheme,but not when binning is not applied. For example, a distant object maynot be identified in a non-binning scheme because the sensor may notreceive enough reflected light to make a positive id of a target object,but the distant object may be detected when a binning scheme is appliedand the received light reflections are considered together.

In other embodiments, some objects may be detected in a non-binningscheme and not detected in a binning scheme. In some cases, processor118 may apply both non-binning schemes and binning schemes to a commonset of sensor pixel outputs and make a determination of which provideshigher quality detection information.

Assigned detection directions to portions of identified target objectsmay occur “off grid” relative to the pixels of a LIDAR sensor when abinning scheme is applied. For example, usual non-binning detections mayoccur, if at all, every 0.2 degrees. In a binning scheme, however, wheregroups of sensor pixel outputs and their corresponding FOV pixels may beconsidered together, it may be possible to determine a detectiondirection that does not correlate with the FOV pixel grid/point clouddata point spacing or lie within a center of any particular FOV pixel.Rather, in some cases, the determined detection direction to a targetobject or a portion of a target object (based on a binning scheme) maybe at 7.3 degrees (or any other value based on the parameters of aparticular set of binned sensor pixel output signals) which may be offthe standard. FOV pixel grid. In some embodiments, the determinedlocation of a target object detected using binning may be in the centerof any FOV pixel, off center from any FOV pixel, on a line separatingFOV pixels, or on an intersection of two lines separating FOV pixels. Insome embodiments, the direction assigned to a binned detection pointcloud point may not necessarily be in the middle of the FOV pixelsaffecting by binning, but rather may be based on an average location ofdetections or a center of gravity of the FOV pixels affected by binningof the corresponding sensor pixel outputs. Additionally, binning can beimplemented in different shapes of sensor pixel outputs corresponding toFOV pixels, not only N×N squares of FOV pixels. In some embodiments,binning can be implemented in a rectangular shape (e.g., any N×Mgrouping), a triangular shape, generally circle shaped, ellipticallyshaped, etc. In some embodiments, binning of sensor pixel outputs may bebased on temporal considerations. For example, as noted above, differentsensor pixel outputs may be combined based on timing (e.g. combiningitems of vectors to provide a binning vector, or adding the analogsignal over time).

In some embodiments, processor 118 may process a stream of sensor pixeloutputs according to unbinned or binned processing schemes. In somecases, as previously noted, whether to bin two or more pixel outputs maybe based on prior detection determinations relative to one or more pixeloutputs. For example, in some cases, processor 118 may initially processa subset of sensor pixel outputs (which are provided as a plurality ofinput signals to the processor) without binning. If no object isdefinitively detected based on analysis of the subset of sensor pixeloutput signals, processor 118 may re-process the subset of sensor pixeloutput signals by combining together at least two sensor pixel outputsignals of the subset according to a determined binning scheme. In someembodiments, processor 118 may process some or all of the sensor pixeloutput signals/processor input signals of the subset according to abinning scheme. Similarly, if at least some of the received sensor pixeloutput signals/processor input signals are determined to be associatedwith or to provide insufficient detection information, the processor mayprocess those sensor pixel output signals/processor input signals bycombining at least two of the input signals according to a binningscheme.

In some embodiments, application of a binning scheme may depend on adetermined confidence level in a detection of a target object. Forexample, processor 118 may initially determine based on analysis of asubset of input signals that a potential target object exists in aregion of the FOV corresponding to the analyzed input signal. The objectdetection, however, may not be associated with a confidence level thatexceeds a predetermined level (e.g., 40%, 60%, 75%, etc.). In asubsequent process, processor 118 may apply a binning scheme such thattwo or more of the input signals are re-processed together.

In some embodiments, more than one binning scheme may be applied to agroup of sensor pixel outputs (e.g., as part of a series of processesperformed relative to a set of input signals). For example, in a firstprocess, processor 118 may process a subset of pixel outputs received asinput signals by applying a first binning scheme (e.g., a non-binningscheme, a 2×2 binning scheme, etc.) relative to the input signals. Sucha process may result in the identification of an existence of a firstobject in a particular region of the LIDAR FOV associated with theprocessed input signals. In some cases, processor 118 may re-process theinput signals according to a different binning scheme (e.g., a 4×4binning scheme, etc.). Such re-processing of the input signals mayresult in the detection of a second object in the particular region ofthe LIDAR FOV associated with the re-processed input signals. In somecases, the second object may be more distant, less reflective, etc. thanthe first object. In other cases, however, the second object may be morereflective, less distant, etc. relative to the first object.Additionally, in some cases, the second object may not be identifiablein the first processing scheme.

In some cases, two or more binned groups of pixel outputs may beconsidered together in order to determine the presence of a targetobject in a LIDAR FOV. For example, in some embodiments, processor 118may combine the results obtained from an application to a group of inputsignals (pixel outputs) of a first binning scheme (e.g., non-binned, 2×2bins, etc.) and application of a second binning scheme (3×3 bins, etc.)in order to identify an object in a region of the LIDAR FOV. In somecases, the identified object may not have been identified throughprocessing of the input signals individually (unbinned) or throughprocessing of the input signals according to the first binning schemealone or the second binning scheme alone.

Any binning scheme may be employed by processor 118 depending on therequirements of a particular application. In some embodiments, a firstprocessing scheme may be applied to a first group of input signalsrelative to a particular LIDAR FOV region, and a second processingscheme may be applied relative to a second group of input signalsrelative to the particular LIDAR FOV. In some cases, the first andsecond processing schemes may be different.

In some embodiments, processor 118 may select a binning scheme based onillumination levels associated with particular regions of the LIDAR FOV.For example, processor 118 may bin together pixel outputs associatedwith FOV regions where less light is projected and may process pixeloutputs individually (or with smaller bins) relative to regions of theLIDAR FOV where more light is projected.

In some cases, binning of sensor pixel outputs may be accomplished bycombining (or considering together) sensor pixel outputs derived fromnon-spatially separated FOV pixels (FOV pixels that share at least oneborder, corner, etc.). In other cases, binning of sensor pixel outputsmay be accomplished by combining (or considering together) sensor pixeloutputs derived from spatially separated FOV pixels (FOV pixels that donot share at least one border, corner, etc.). Not all pixel outputsreceived from the LIDAR sensor relative to a particular LIDAR FOV needbe binned if a binning scheme is applied. Rather, in some cases, abinning scheme may be applied to only a subset of sensor pixel outputsassociated with particular regions of FOV pixels of the LIDAR FOV. Othersensor pixel outputs corresponding to other regions or FOV pixelsassociated with the same LIDAR FOV may remain unbinned. For example, insome embodiments, processor 118 may apply a non-binning scheme forsensor pixel outputs associated with objects in near field region andmay apply a binning scheme for pixel outputs associated with objects infar field region. In some cases (e.g., where a near field and far fieldoverlap or where pixel outputs may represent both close in andbackground portions of a scene), some sensor pixel outputs within aparticular set of input signals to the processor may be used for bothnon-binning and binning schemes.

Binning schemes may also be applied based on other types of feedback.For example, in some cases, a region of interest (ROI) may be identified(e.g., a region occurring in a particular range relative to the LIDARsystem, a region where target objects of a particular type are expected,a region where a past object detection occurred, etc.). A binning schememay be selected and applied relative to a particular LIDAR FOV region orsubregion based on whether such a region or subregion includes an ROI.

Any suitable bin size can be used depending on the requirements of aparticular application. In some cases, the bin size may be based on thesize of a detected object (or expected object) relative to a LIDAR FOVregion (e.g., how many FOV pixels a particular object at a particularrange is expected to occupy). In some cases, regions of the FOV to whicha binning scheme is applied may be predefined, and the sizes of thoseregions may be predefined. In other cases, the sizes of the regions towhich a binning scheme is to be applied may be determined dynamically.For example, a region of a LIDAR FOV to which binning is applied or notapplied to associated sensor pixel outputs may be predefined based onwhether detected objects are within 50 m from the LIDAR system, arefarther than 50 m from the LIDAR system, etc. Binning schemes may alsobe employed based on a detected driving environment. For example, whenthe processor determines that a host vehicle is in a rural environmentor an urban environment, processor 118 may employ predefined binningschemes—at least one binning scheme associated with a detected ruralenvironment, and at least one different binning scheme associated with adetected urban environment. In some cases, binning may be predeterminedbased on a region of the LIDAR FOV. For example, in some cases, allsensor pixel outputs associated with the FOV pixels in an upper third ofa LIDAR field of view may be processed according to at least onepredefined binning scheme, while sensor pixel outputs associated withthe FOV pixels in the other two thirds of the LIDAR FOV may be processedaccording to a different binning scheme (e.g., unbinned or binnedaccording to a different size, shape, etc.). The size of an FOV regionto which a binning scheme is to be applied by combining at least twosensor outputs associated with that region may be determined dynamicallybased on at least one of detected ambient lighting conditions, objectreflectivity, noise levels, vehicle speed, and/or driving environment.

In some embodiments, processor 118 may dynamically define a size orshape of a bin relative to a LIDAR FOV region and associated pixeloutputs. Such a definition may be based on a variety of factors (e.g.,detected road conditions, changes in reflection signals received,ambient lighting conditions, object reflectivity, detected noise levels,detected vehicle speed, detected driving environment, etc.). Theapplication of a binning scheme may also depend on a detected horizonlocation. For example, in some cases, sensor pixel outputs associatedwith regions above a detected horizon may be binned while thoseassociated with FOV regions below a detected horizon may be processedaccording to a different binning scheme (unbinned, different size orshape of bins, etc.). Binning of sensor pixel outputs may also depend onrange values relative to the LIDAR system. For example, in a ruralenvironment, up to 25 m in front of the LIDAR system may be considered aforeground and outputs of sensor pixels acquiring light from regions ofthe LIDAR FOV within this range may be unbinned. Ranges greater than 26m, for example, may be considered background and associated sensor pixeloutputs may processed by binning.

In some embodiments, a binning scheme may be applied to conservecomputational resources. For example, processor 118 may implement abinning scheme relative to all or a portion of the sensor pixel outputsassociated with an entire scan of the LIDAR FOV. Such binned processingmay require less computational resources as compared to processing ofall of the sensor pixel outputs individually. Based on the results ofthe binning process, however, processor 118 may re-process any of thesensor pixel outputs for the acquired scan of the LIDAR FOV according todifferent binning scheme (e.g., no binning, different sizes or shapedbins selectively applied to different ROIs, for example). For example,if a host vehicle is determined to be traveling on a rural road,processor 118 may implement a binning scheme for 50%, 75%, 90%, 100%,etc. of the sensor pixel outputs associated with the LIDAR field ofview. If, for example, processor 118 detects an object on the ruralroad, processor 118 may unbin sensor pixel outputs or implement a newbinning scheme relative to those sensor pixel outputs corresponding to aportion of the LIDAR FOV determined to include the detected object. As aresult, a higher resolution of LIDAR distance data may be obtainedrelative to the detected object (e.g., a point cloud with a higherdensity of data points), but the higher resolution, more computationallydemanding process may be implemented as needed based on objectdetections. Such a technique may conserve computational resources.

In some cases, sensor pixel outputs may be binned across different scansof a LIDAR FOV. For example, unbinned or binned sensor pixel outputsacquired relative to a first scan of a LIDAR FOV may be combined (orconsidered together) with unbinned or binned sensor pixel outputsacquired relative to at least one subsequent scan of the LIDAR FOV.

In some embodiments, illumination of certain regions of a LIDAR FOV maybe related to an applied binning scheme. For example, processor 118 mayprocess sensor pixel outputs associated with FOV pixels in a firstregion of a LIDAR FOV that receive a certain illumination levelaccording to a first binning scheme (e.g., unbinned, 2×2 bins, etc.) andmay process sensor pixel outputs associated with FOV pixels in a secondregion of the LIDAR FOV that receive a different illumination levelaccording to a different binning scheme. The illumination level may bedetermined based on the binning scheme employed. Conversely, the binningscheme may be determined based on the illumination scheme employed. Insome cases, the binning scheme applied to the second region may includebins larger than bins applied relative to the first region. Based on theapplied binning scheme, processor 118 may control a LIDAR light sourceto illuminate the FOV pixels of the first region of the field of view ata first illumination level and illuminate the FOV pixels of the secondregion of a field of view at a second illumination level lower than thefirst illumination level. For example, the binned scheme applied to thesecond region, including larger binned groups than used relative to thefirst region, may not require as high of an illumination level. In otherembodiments, processor 118 may control the LIDAR light source toilluminate the FOV pixels of the second region of the field of view atan illumination level higher than an illumination level applied to theFOV pixels of the first region. For example, a more distant objectlocated in the second region may benefit from a higher illuminationlevel for greater confidence of detection.

FIG. 21 provides a flow chart of an exemplary method for using a LIDARsystem to determine distances to objects in a field of view. At step2101, the method may include controlling at least one light source forilluminating the field of view. At step 2103, a plurality of inputsignals indicative of reflections of light from objects in the field ofview may be received by processor 118 from a group of detectors (e.g.,pixels), for example. At step 2105, processor 118 may process a firstsubset of the input signals associated with a first region of the fieldof view to detect a first object in the first region, wherein processingthe first subset is performed individually on each of the first subsetof the input signals. At step 2107, processor 118 may process a secondsubset of the input signals associated with a second region of the fieldof view to detect at least one second object in the second region,wherein the at least one second object is located at a greater distancefrom the light source than the first object and wherein processing ofthe second subset includes combining input signals of the second subset.At step 2109, processor 118 may output information associated with adistance to the first object and information associated with a distanceto the at least one second object.

Distributed LIDAR System

LIDAR systems (such as the LIDAR systems described above) may bedeployed on any platform where distance ranging information to objectsin a platform environment may be useful. In some cases, for example, thedisclosed LIDAR systems may be deployed on a vehicle to provide ranginginformation relative to objects in an environment of the vehicle. Forexample, in some cases, it may be desirable to provide one or moreoptical gateways to the LIDAR system (e.g., deflector elements that mayboth project laser light to an environment of the vehicle and receivereflected laser light from the environment of the vehicle) at locationsaround a perimeter of a vehicle envelope that may offer relativelyunimpeded field of views to areas of interest within a vehicleenvironment. Such locations may include, for example, areas in and/oraround the front, sides, roof, or rear of a vehicle (e.g., on or withina vehicle front or rear bumper, grill, headlight assembly, taillightassembly, or any other fixture associated with the vehicle).

In many cases, however, such vehicle locations may present certainchallenges to LIDAR system deployment. For example, in some situations,there may be insufficient space available for a LIDAR system, includinga light source, deflector, processor, etc. at a desired location on avehicle perimeter. In other cases, sufficient space for a complete LIDARsystem may be available in desired locations around a vehicle perimeter,but placing the LIDAR system in such regions may foreclose placement ofother potential systems or system components in those locations.

Additionally, certain locations on a vehicle, such as near an enginecompartment, exhaust system, etc., may impact heat dissipationcapabilities within those regions. With respect to LIDAR systemdeployment, placing heat-generating components of a LIDAR system (e.g.,various processors associated with the LIDAR) within high-heat regionsof a vehicle (or other type of platform) may impact an ability of theLIDAR system to effectively dissipate heat generated by thesecomponents. Excess heat in these systems can negatively impact theoperation of one or more LIDAR system components. For example, asdescribed above, the LIDAR deflector may include one or more scanningmirrors that hinge on tiny bendable hinges in order to control thedirection of light projection and/or light collection. The operation ofthe deflectors may be degraded in situations where excess thermal energymay cause changes in one or more of their operational properties.

Both of these challenges as well as others may be addressed withdistributed LIDAR system embodiments described herein and may providefurther advantages. For example, as will be discussed in detail below,the disclosed LIDAR system may be distributed such that a front-end unitof the system, while not necessarily related to the front of a vehicle,may provide an optical gateway to an environment of the LIDAR system ofthe vehicle (e.g., at least a deflector for projecting laser light to anenvironment of interest), and a separate back-end unit, spaced apartfrom the front-end unit and not necessarily related to or located nearthe back of the vehicle, that may include other components of the LIDARsystem (e.g., one or more LIDAR processing units). It should be notedthat the terms front-end unit and back-end unit, in general, are used todistinguish between distinct portions of a distributed LIDAR system anddo not indicate a relationship with a particular location on a vehicle.

Such a distributed LIDAR system may provide various potential benefits.For example, by separating one or more processing units of the LIDARsystem from optical front-end components, the distributed system mayoffer increased flexibility in where the front-end components areplaced. For example, in some cases, the optical front-end components maybe located in tight spaces at the front, side, roof, or rear of avehicle that may be too small to accommodate a complete LIDAR system.Additionally, the separation of one or more processing units of theLIDAR system from optical front-end components in the distributed systemmay offer enhanced heat management capabilities (e.g., by thermallyisolating the front-end unit(s) from the back-end unit(s)). For example,processing units associated with a LIDAR back-end unit may be moved awayfrom heat generating locations of the vehicle (e.g., engines, exhaust,etc.) and may be located in other locations where heat dissipation maybe less challenging. Additionally, separating heat-generating LIDARcomponents from one another, such as the separation of processing unitsand/or light sources from light deflectors, may reduce the sensitivityof the LIDAR system to thermal variations and may enhance performance.Additionally, separating heat-sensitive LIDAR components fromheat-generating LIDAR components from may improve the performance of theheat-sensitive LIDAR components. The disclosed systems may also includedata conduits and/or light conduits for providing connectivity betweenthe LIDAR front-end and back-end units.

The disclosed systems may have a variety of configurations. In oneexample, the disclosed systems may include one or more LIDAR front-endunits, one or more LIDAR back-end units, and a data conduit. In somecases, the conduit connecting the front-end and back-end units may alsoinclude optical components to transfer light from one component toanother (e.g., laser light for projection from the back-end to thefront-end and/or received reflected light from the front-end to theback-end for sensing). The front-end units may be located remotely fromthe back-end unit, which may reduce or eliminate heat impact of theback-end components on components housed in the front-end units, forexample. In some embodiments, the back-end unit may include a processingunit with a processor and the front-end units may include a projectingunit including a scanning unit with a light deflector and an actuator.One or more light sources may be located in the front-end units or theback-end units.

FIG. 22 is a diagram showing an exemplary LIDAR system with distributedLIDAR system components including first housing 2230, which may also bereferred to as a back-end unit, a plurality of second housings 2210, forexample 2210A-D, which may also be referred to as a front-end units,data conduits 2220, for example 2220A-D, processor 118, controllablelight deflectors 114, light sources 112, and actuators 302.

Back-end unit 2230 may include any structures or components suitable forconducting the main processing of the LIDAR system. Back-end unit 2230may have a variety of configurations and may include various components.In one example, back-end unit 2230 may include a processing unit and oneor more processors, such as processing unit 108 and processor 118 ofFIG. 1A. As depicted in FIG. 1A, processing unit 108 may includeprocessor 118. In some embodiments, processor 118 may control componentsof front-end units 2210. For example, as depicted in FIG. 22, processor118 may coordinate operation of light source 112 with the movement oflight deflector 114 of FIG. 1A (e.g., by actuator 302 of FIG. 3A) inorder to scan a field of view and receive light reflected from objectsin the field of view. Among other potential benefits, locatingprocessing unit 108 within back-end unit 2230 rather than includingseparate processing units for each front-end unit 2210 may increaseefficiency through centralization, as a single processing unit mayreceive, process, and control information from multiple front-end units.For example, as shown in FIG. 22, processor 118 may receive informationvia data conduits 2220 from four different front-end units, e.g., theplurality of four front-end units 2210A-D of FIG. 22. Each data conduitmay be configured to interconnect the first housing (back-end unit) andat least one second housing (front-end unit). Each data conduit may beassociated with a forward path from the back-end unit to the at leastone front-end unit and a return path from the at least one front-endunit to the back-end unit. Each data conduit may be configured tocooperate with the at least one processor and at least one actuatorassociated with a light deflector, for example, such that the forwardpath is enabled to convey control signals for controlling the at leastone actuator, and the return path is enabled to convey to the at leastone processor reflections signals indicative of light reflected fromobjects in the field of view. In some embodiments, there may be separateback-end units for controlling and for processing.

As noted above, processing units associated with the disclosed LIDARsystems (among other components) may generate heat, and such heat mayaffect other components, such as the light deflectors responsible fordirecting laser light to an environment and/or for collecting reflectedlight from the environment. In some embodiments, as depicted in FIG. 22,processor 118 may be separated from other LIDAR system components. Forexample, processor 118 may be placed in back-end unit 2230, while otherLIDAR components, such as deflectors 114 may be placed in front-endunits 2210. Such separation may mitigate potential effects of heat fromLIDAR system processors on other LIDAR components. Furthermore,distribution of LIDAR system components may enable selective locationand advantageous positioning within a platform (e.g., a vehicle) ofcertain LIDAR components. In some embodiments, it may be desirable forthe LIDAR system to have an optical gateway at the front of a vehicle(e.g., in order to scan an FOV forward of the vehicle). Many vehicles,however, include an engine unit at the front of the vehicle, which cancontribute to significant levels of heat in this region. Thus, in somecases, placement of a full LIDAR system near a front region of a vehiclemay exacerbate heat effects on the LIDAR system. For example, heatgenerated by LIDAR processing units may be difficult to dissipate inhigh-heat regions near a vehicle engine, which can lead to adverseimpact on LIDAR components such as the deflectors or otherheat-sensitive components. Moreover, some locations may be considered“prime” locations, e.g., interesting for camera, LIDAR, vehiclecomponents, etc., so it may be advantageous to minimize LIDAR footprintin such prime locations. The distributed LIDAR systems of the presentembodiments, however, may enable separation of back-end unit 2230,including one or more LIDAR processors, from high-heat areas of avehicle, such as near an engine. For example, in some embodiments,back-end unit 2230 may be located near a center of the vehicle, in arear portion of the vehicle, in a passenger compartment of the vehicle,on a roof of the vehicle, etc. Back-end unit 2230 may be locatedanywhere in the vehicle. As depicted in the exemplary system shown inFIG. 22, back-end unit 2230 can be located in the rear of the vehicle.Alternatively (e.g., where a vehicle has a mid-engine or rear-enginedesign), back-end unit 2230 may be located in the front of the vehicle,or in another location away from the engine. In some embodiments,processor 118 may be placed in front-end unit 2210.

In some embodiments, LIDAR system 100 in a vehicle (e.g., vehicle 110 ofFIG. 6A) may be configured with one or more processors (e.g., processors118A, 118B, and 118C, or any other LIDAR processor) located in aback-end unit, e.g., back-end unit 2230. In such embodiments, theplurality of front-end units (e.g., units 2210A-D of FIG. 22) mayinclude projecting units 102 separated from the back-end units. Such aconfiguration may separate the processing unit components from at leastthe projecting unit components of the LIDAR system 100 (e.g., any of thedeflector, actuator, and/or light source). In some embodiments and asdepicted in FIGS. 22 and 24, at least one light source 112 of FIG. 1Amay be located in one or more front-end units. In other embodiments andas depicted in FIG. 23, at least one light source 112 of FIG. 1A may belocated in the back-end unit.

In some embodiments processor 118, which may be located in back-end unit2230, may be configured to synchronize an operation of a plurality oflight deflectors and a plurality of actuators in the plurality offront-end units. Any combination of the following functions may beimplemented by back-end unit 2230: monitoring and controlling thedeflector (and other components), processing data, and decision making(in LIDAR/host level). For example, in one embodiment, processor 118 ofback-end unit 2230 may send control signals via data conduit 2220A anddata conduit 2220B to actuators 302 of front-end units 2210A and 2210Bto move light deflectors 114 simultaneously. In some embodiments,processor 118 may simultaneously move light deflectors 114 in allfront-end units. In some embodiments, processor 118 may simultaneouslymove light deflectors 114 in front-end units located in the rear of thevehicle. In other embodiments, processor 118 may simultaneously movelight deflectors 114 in front-end units located in the front of thevehicle. In other embodiments, processor 118 may simultaneously movelight deflectors 114 in front-end units located on the roof of thevehicle. In yet other embodiments, processor 118 may simultaneously movelight deflectors 114 in front-end units located in the side of thevehicle.

In some embodiments, processor 118 may be configured to change theselection of which of the various front-end units 2210 are active (e.g.control deflector by actuator to scan the incident light beam over thefield of view). For example, processor 118 may be configured to changethe selection of which of the various front-end units 2210 are activebased on detection information. For example, a processor associated withthe LIDAR system and located in a back-end unit may selectively controloperation of one or more front-end components (e.g., deflector,actuator, sensor, and/or light source) based on LIDAR detection relativeto the host vehicle environment or based on any other source providinginformation regarding objects in an environment of the host vehicle(e.g., vehicle radar, vehicle mounted cameras, sensors onboard orlocated remotely from the host vehicle, etc.). As an illustration, if apedestrian crosses in front of the vehicle and moves outside the presentfield of view of front-end unit 2210B, but the pedestrian may bedetected by another front-end unit, e.g. front-end unit 2210A, thenprocessor 118 may use front-end unit 2210A to scan the field of view. Inanother example, if another vehicle changes lanes in front of the hostvehicle and moves outside the present field of view of a front-end uniton the side of the vehicle, but the other vehicle may be detected byanother front-end unit, e.g. on the front of the vehicle, then processor118 may use the front-end unit on the front of the host vehicle to scanthe field of view. In yet another example, the LIDAR system may includea plurality of forward-looking front-end units (directed towards thefront of the vehicle) and a plurality of backward-looking front-endunits (directed towards the front of the vehicle). The processor maychoose to utilize all of the front-end units directed in the directionof propagation (e.g., driving forward or in reverse gear), and only oneof the front-end units directed in the opposite direction.

In another example, a back-end processor may alter one or moreperformance characteristics associated with at least one front-endcomponent based on information feedback. For example, a back-endprocessor (e.g., processor 118) may be configured to use informationobtained from reflections signals sensed by any of the front-end unitsto cause a change in light flux projected from the same and/or any oneof the other front-end units. For example, in response to receivinginformation from front-end unit 2210B via data conduit 2220B, forexample, identifying a stopped car ahead, processor 118 of back-end unit2230 may send a control signal via data conduit 2220A to actuator 302 offront-end unit 2210A to cause a change in the light flux projected fromfront-end unit 2210A in advance of changing lanes.

In some embodiments, processor 118 may be configured to identify asubgroup of the plurality of front-end units 2210 for processing ofreflections signals based on an operational condition of the vehicle.For example, when the host vehicle is driving in reverse, processor 118may identify a subgroup including front-end unit 2210D and front-endunit 2210C which are located in the rear of the vehicle and there accessto fields of view behind the host vehicle. In another embodiment, whenthe vehicle is driving forward, processor 118 may identify a subgroupincluding front-end unit 2210A and front-end unit 2210B which arelocated at the front of the host vehicle and their access to fields ofview forward of the host vehicle. Based on such determinations, theprocessor may selectively activate or control various front-endcomponents. Additionally or alternatively, the processor may alter oneor more operational characteristics of any of the components of anidentified subgroup of front-end units. For example, light flux providedby any of the front-end units may be increased or decreased, scanningrates and/or scanning patterns may be changed, etc.

While light sensitive components for sensing reflected laser light maybe located in the front-end units of a LIDAR system, they need not be.For example, such sensors may also be located in LIDAR back-end units.FIG. 23 provides a diagrammatic view of an exemplary LIDAR system withdistributed LIDAR system components in which the back-end unit includesat least one sensor 116. As shown in FIG. 23, the LIDAR system mayinclude back-end unit 2230, a plurality of front-end units 2210E-G, anddata conduits 2220E-G. Processor 118 may be located in back-end unit2230 along with one or more sensors 116, which may function as acentralized sensor for detecting laser light reflections collected, forexample, by light deflectors 114 located in the front-end units 2210E,2210F, and 2210G. In such embodiments, collected laser light may betransferred from a front-end unit to a back-end unit via any suitablemeans. For example, in some cases, one or more fiber optic conduits maybe included in or separate from data conduits 2220E, 2220F, and 2220G.Similar arrangements may be used in embodiments that include acentralized laser light source or a plurality of centralized lasers inorder to distribute laser light for projection to an environment of thehost vehicle via deflectors included in one or more of the availablefront-end units of the LIDAR system. In some embodiments, in front endunit 2210, the RX channel may be optically combined with the TX channel.In other embodiments, in front end unit 2210, the RX channel may not beoptically combined with the TX channel.

In some embodiments, processor 118 may be configured to use informationobtained from reflections signals associated with one of the pluralityof front-end units 2210E-G to cause a change in sensitivity of a sensorlocated in back-end unit 2230 (or at any other location within the LIDARsystem, e.g., in a front-end unit). For example, in response toreceiving information from front-end unit 2210G via data conduit 2220G,processor 118 of back-end unit 2230 may adjust sensitivity of a sensor116 located in back-end unit 2230.

In some embodiments, back-end unit 2230 may include at least onevibration sensor configured to determine data indicative of vibrationsof the vehicle. In other embodiments, back-end unit 2230 may include alocation sensor, velocity sensor, acceleration sensor, and directionfeedback sensor. In such embodiments, in response to informationreceived from the any one of the sensor, for example, processor 118 maysend control signals via data conduits to an actuator, light source,etc. located in one or more front-end units to cause a change in thelight flux projected from any of the one or more front-end units. If,for example, there is a vibration based on the conditions of the road(pothole, rough road, etc.), the scanning properties of the deflectormay be adjusted. In one example, processor 118 may adjust the light fluxprojected from any of the front-end units based on detected vibrations.

In some embodiments and as depicted in FIG. 23, back-end unit 2230 mayinclude light source 112 of projecting unit 102 of FIG. 1A. Light source112 of projecting unit 102 of FIG. 1A, like processor 118 describedabove, may generate a significant amount of heat. Combining a lightsource 112 and processor 118 of FIG. 1A into one housing and separatingthem from the deflectors may improve heat dissipation and may reduce oravoid heat related drift of the light deflectors located in thefront-end units.

As previously noted, in some embodiments, data conduits or separatelight conduits may be configured to convey the light from a light sourcelocated in a LIDAR back-end unit to any of the available LIDAR front-endunits in order to project the light to various sections of theenvironment of the host vehicle. In some embodiments, the data conduit(or separate light conduit) may include fiber optics, one or moreoptical elements establishing an optical path (e.g., one or more lenses,filters, mirrors, etc.) to convey light from a back-end light source tofront-end components.

Front-end units 2210 may have a variety of configurations and mayinclude various components. In one example and in reference to FIG. 22,front-end units 2210 may include projecting unit 102 and light source112 of FIG. 1A, scanning unit 104 and light deflector 114 of FIG. 1A,and actuator 302 of FIG. 3A. In some embodiments, front-end units 2210may further include one or more ancillary processors, but not the mainLIDAR analysis processor, e.g. processor 118 of back-end unit 2230,which may be used to detect and classify objects in the field of view asdescribed above.

Front-end units 2210 may be located in the host vehicle at any suitablelocation. As described above, the front-end units may be remotelylocated relative to a LIDAR back-end unit. In some embodiments,front-end units 2210 may be located on the side of the vehicle for sideimaging, e.g., identifying and measuring distances to various locationson objects located on the side of the vehicle and generating anassociated point cloud based on the collected distance information. Insome embodiments, front-end units 2210 may be located on the front ofthe vehicle for forward imaging. In some embodiments, front-end units2210 may be located on the rear of the vehicle for rear imaging. In someembodiments, front-end units 2210 may be located on the roof of thevehicle for higher vantage point imaging and/or 360 degree imaging (orany portion of 360 degrees). In some embodiments, one or more of theplurality of front-end units 2210 may be located on or behind thewindshield of the host vehicle. In some embodiments, one or more of theplurality of front-end units 2210 may be located in the head lights ortail lights (or their associated housings) of the vehicle, within aforward or rear bumper, within the vehicle grill, coupled with anantenna, etc. A combination of positions of front-end units 2210 (e.g.,in a plurality of the above-identified example locations) may enable alarger aggregated field of view of an environment of the host vehicle.In some embodiments, front-end units 2210 or a component withinfront-end units 2210 may be rotatable so as to change a FOV of the unit(e.g. rotating/moving the FOV 90° to the left, right, up, or down).

Separation of the LIDAR components into one or more front-end units andat least one back-end unit may provide flexibility in how a LIDAR systemis deployed on a particular platform. For example, where LIDARprocessing units, light sources, and/or sensors are located in a LIDARback-end unit, accompanying LIDAR front-end units may need to housefewer LIDAR components, which may enable the front-end units to belocated in smaller spaces. In some embodiments, a LIDAR front-end unitmay include a LIDAR deflector and actuator. In other embodiments, theLIDAR front-end may include a LIDAR deflector, an actuator, and asensor. Such front-end units may include accompanying components aswell. In such embodiments, the LIDAR front-end units may occupyrelatively small volumes, which may enable their placement in suchlocations as behind a rearview mirror of a vehicle, within a headlightassembly, within a vehicle grill, within or on a vehicle bumper, withinor on a vehicle antenna assembly, within or on a vehicle door handle, orany other low volume location. In some cases, e.g., where a front-endunit includes a light deflector, actuator, and optionally a sensor, theLIDAR front-end unit may be incorporated at nearly any location on or ina host vehicle.

By distributing LIDAR system components between front-end and back-endunits, in some embodiments, each of the plurality of front-end units(e.g., units 2210) may occupy a smaller volume than a LIDAR back-endunit of the same system (e.g., back-end unit 2230). In some embodiments,back-end unit 2230 may be substantially larger in size than any of theplurality of front-end units. For example, a back-end unit may occupy agreater volume than a front-end unit. In some cases, a LIDAR back-endunit may have volume that is 1.2×, 2×, 5×, 10× 25×, 50×, 100× or morelarger than a LIDAR front-end unit deployed as part of a common LIDARsystem. While any dimensions for the front-end and back-end units may beused depending on the requirements of a particular application, in oneexemplary embodiment, a LIDAR front-end unit may have a total volume ina range of about 80 cm3, and a LIDAR back-end unit may have a totalvolume in a range of about 500 cm3.

The disclosed LIDAR systems may include any number of front-end andback-end units depending on the requirements of a particularapplication. In some cases, the disclosed LIDAR systems may include onefront-end unit and one back-end unit. In other cases, the disclosedLIDAR system may include multiple front-end units (2, 4, 8, 10, 25 ormore) together with a single, central back-end unit. In still othercases, a LIDAR system may include multiple front-end units as well asmultiple back-end units. In embodiments that include multiple front-endunits, the front-end units may be deployed, for example, at variouslocations around a host vehicle for an improved or increased field ofview.

As described above, a controllable light deflector (e.g., such asdeflector 114) of a LIDAR front-end unit may include any structures orcomponents suitable for directing light to a LIDAR field of view. Insome embodiments, the light deflector may include a MEMS mirror. In someembodiments, the light deflector may include an optical phased array(OPA), a mechanical mirror, a rotatable polygon prism, a crystal, or anyother form of controllable deflector. A scanning unit of a front-endunit may be either bidirectional or one-directional, scanning only theoutgoing illumination, scanning only the incoming light reflections, orscanning in both directions (e.g., as depicted in FIG. 2A). A scanningunit of a front-end unit may be either one-dimensional (e.g., scanningonly across one axis) or two-dimensional (e.g., scanning across twoaxes). A single mirror may be deployed in a front-end unit and may beused to deflect light to the field of view and deflect light from thefield of view to the sensor. In other cases, two or more differentmirrors may be provided in a front-end unit to project and deflectlight. In other cases, a front-end unit may include one or more mirrors(or other type of deflector) to project light toward a field of view,and one or more different mirrors (or other type of deflector) tocollect reflected light from the field of view. Any of the opticalchannels available within a particular front-end unit may be selectivelyenabled by the LIDAR processor, for example, in response to detectionfeedback or other available information. Each deflector deployed in afront-end unit may include one or more light deflectors. In someembodiments, each light deflector associated with each front-end unitmay include an array of light deflectors. For example, light deflector114 of front-end unit 2210B may include an array of four lightdeflectors (e.g., light deflectors 114A-D). In some instances, afront-end unit may include an array of 2, 5, or 10 (or more) lightdeflectors. In some cases, the front-end unit may even include an arrayof 100 or more light deflectors.

As described above, a plurality of LIDAR front-end units may be deployedat locations around a vehicle envelope such that two or more of theLIDAR front-end units offer overlapping fields of view. In such cases, aLIDAR processor (e.g., processor 118) may be configured to controlmovement of a plurality of light deflectors within one or more LIDARfront-end units such that an accumulated energy density of projectedlight in at least one overlapping region is under a maximum permissibleexposure. As noted, overlapping regions of a field of view may includeregions where light from more than one deflector is provided.

While in the disclosed distributed LIDAR systems, a LIDAR processor(e.g., processor 118) may be disposed within a back-end unit, such aprocessor may be used to control one or more components located in oneor more of the front-end units. For example, in some embodiments aprocessor in a back-end unit may be used to control the operation of anactuator (e.g., actuator 302) in order to cause an associated deflectorto scan light over a LIDAR FOV or at least a portion of a LIDAR FOV. Aback-end unit processor may be used to control any number of componentsincluded in one or more front-end units. For example, in someembodiments, a single processor included in a central back-end unit maybe used to control multiple components across multiple front-end units.In one example, a back-end unit processor may send control signals tomultiple front-end unit actuators (e.g., actuators 302 of front-endunits 2210A and 2210B) in order to cause multiple light detectors acrossdifferent front-end units to move simultaneously. In some cases, suchmovement may be substantially synchronized such that scans of multiplefields of view associated with multiple deflectors are completed withinsimilar time intervals. Such control signals may be conveyed from theback-end processor(s) to one or more actuators (or other components) ofone or more front-end units via data conduits (e.g., data conduits 2220Aand 2220B.

FIG. 24 provides a diagrammatic representation of another exemplaryembodiment consistent with the present disclosure. As depicted in FIG.24, front-end units 2210 may include a sensing unit 106 including asensor 116. As in other embodiments of the distributed LIDAR system, theprocessing unit 108 and processor 118 may be deployed within back-endunit 2230. The plurality of front-end units 2210 include at least onesensor 116, e.g., sensors 116A-116D, for capturing and measuring lightreflected from objects in the field of view. In some embodiments,front-end units 2210 may each include multiple sensors. Reflectionssignals generated by the light sensors may include data that is conveyedto the at least one processor via return paths provided by the dataconduits connecting the LIDAR front-end unit(s) with the LIDAR back-endunit(s).

In other embodiments (e.g., where a light sensor is located in a LIDARback-end unit), the reflection signals transmitted by the dataconduit/optical conduit connecting a LIDAR front-end unit with a LIDARback-end unit may include reflected light collected by one or moredeflectors or other optical components associated with a LIDAR front-endunit. In such cases, the reflections signals include the reflected lightthat is conveyed from the at least one front-end unit to a back-end unitvia the return path. In such embodiments, the data conduits may beconfigured to convey light and may include any structure suitable forlight transmission (e.g., optical fiber cables, coaxial cables,waveguides, lenses, light guides, mirrors, etc.) As noted, a return pathof a data conduit, e.g., from each of the plurality of front-end units2210 to back-end unit 2230, data conduits 2220 may convey reflectionssignals including either the light itself or data from the light.

In some embodiments, processor 118 may be further configured to useinformation obtained from reflections signals associated with one ormore of a plurality of front-end units to cause a change in asensitivity of a sensor located in another of the plurality of front-endunits. For example, in response to receiving information from front-endunit 2210D via data conduit 2220D, processor 118 of back-end unit 2230may adjust sensitivity of sensor 116C located in front-end unit 2210C.In another embodiment, processor 118 may use information obtained fromreflections signals associated with one of the plurality of front-endunits to cause a change in a sensitivity of sensors located in all ofthe plurality of front-end units, e.g., sensors 116A-D of front-endunits 2210A-D.

Data conduits 2220 may include any structures or components suitable fortransferring information from one location to another. In someembodiments, data conduits 2220 may include connectors that interconnectback-end unit 2230 and the plurality of front-end units 2210. Forexample, data conduits may include various types of connections. In someexamples, data conduits 2220 may include CANDL communications, wired,optical fiber, a coaxial cable, a waveguide, or any cable capable ofconveying light, instructions, and information. The type of data conduitmay depend on the type of data the data conduit is transferring.

A data conduit may be configured to cooperate with the processor andactuator such that the forward path is enabled to convey control signalsfor controlling the at least one actuator. According to someembodiments, in a forward path of a data conduit, e.g., from back-endunit 2230 to each of the plurality of front-end units 2210, dataconduits 2220 may convey light (e.g., from a LIDAR light source). Dataconduits 2220 may also convey instructions from processor 118 toactuators 302 in the forward path or to any other controllablecomponents of the front-end unit(s). Such control signals, for example,may alter one or more operational parameter values associated with afront-end component (e.g., a scanning rate, scanning pattern, scanningrange, etc. associated with a deflector included in a front-end unit).In some cases, control signals may cause a direct response in one ormore electrical components associated with the actuator (e.g., a piezoelectric device, etc.) or any other component of front end unit 2210. Inother cases, the control signals may be more complex. For example, theymay include commands or instructions for a processor. Upon receipt, aprocessor associated with the actuator may interpret the control signalsand issue one or more signals to cause a change in the actuatorcomponents. In some instances, control may be electronic signal to causeresponse by actuator. In some embodiments, control signals can be morecomplex instructions/commands (e.g., for execution by a processorassociated with the actuator). In some embodiments, in a return pathcontrol signals may be sent to the back-end unit from a front-end unit.In some embodiments, the return path may be enabled to convey to the atleast one processor reflections signals indicative of light reflectedfrom objects in the field of view.

In some embodiments, each front-end unit 2210 may be different fromanother front-end unit 2210. For example, in the same LIDAR system,front-end units 2210 may include different FOVs, different scanningrates, different illumination power, and different illumination range.Additionally, in some embodiments, some front-end units 2210 may includeone or more sensors while other front-end units 2210 may not. Moreover,in some embodiments, some front-end units 2210 may include one or morelasers while other front-end units 2210 may not. For example, front-endunits 2210A-B of FIG. 24 may have greater FOV, higher scanning rate,higher illumination power, and higher illumination range than that offront-end units 2210C-D due to the position of front-end units 2210A-Bin the vehicle.

While a host vehicle has been described as an example of a platform uponwhich the disclosed LIDAR systems may be deployed, the disclosed LIDARsystems may also be deployed on various other types of platforms. Forexample, in some embodiments, LIDAR system 100 may be included as partof a security camera system (e.g., a non-scanning system) as shownassociated with the scene of FIG. 6D. In this example, the surveillancesystem may include a single rotatable LIDAR system 100 to obtain 3D datarepresenting field of view 120 and to process the 3D data to detectpeople 652, vehicles 654, changes in the environment, or any other formof security-significant data. In some embodiments, the security systemmay include distributed system components, such as any of thosedescribed above. For example, the security system may include a back-endunit, as the processing hub, and may be separated from a front-end unitas the sensing and scanning units. The data conduits may connect thefirst and front-end units in the security system. By implementing thedistributed system component scheme in the security camera, theprocessing of the reflections signals is done in a central location.

Multilayered MEMS Scanning Device

MEMS systems (such as those discussed below, e.g., with respect to theassociated drawings) may include MEMS mirrors, scanning units 104, lightdeflectors 114, and other components. Such MEMS systems may be used inconjunction with a LIDAR system (e.g., any of the LIDAR systemsdisclosed herein), in other types of optical and/or electro-opticalsystems, in sound systems, in sensors, cameras, medical devices, or anyother types of systems. Many MEMS systems are manufactured assingle-wafer devices and, as a result, are limited to a single planargeometry. Such systems are also limited to the mechanical propertiesderivable from the single-wafer.

In some cases, such as the LIDAR systems described herein, MEMS mirrorshaving diameters (or other relevant dimension) on the order of about 5mm or more may be employed. Such systems may also involve significantmovement of the MEMS mirror (e.g., displacement amplitudes/angularrotation) in multiple axes in order to scan a FOV. In order to scan anFOV at useful frame rates for use in various applications (e.g.,vehicular applications in which an FOV may be scanned multiple times persecond), the MEMS mirror may be required to quickly move through aseries of instantaneous positions at high operational frequencies.

Traditionally MEMS mirrors fabricated, for example, from a single waferof silicon may significantly limit the functionality of certain scanningsystems, such as LIDAR systems. For example, scanning systems includinglarge MEMS movable structures (e.g., a MEMS mirror with a diameterlarger than 5 mm) with significant displacement amplitudes/angles maygenerate significant stress relative to the mirror structure (e.g.,silicon). Additionally, a MEMS mirror having a full thickness of thewafer material from which it was manufactured (or even the fullthickness of the one or more relevant layers of the wafer) may have amass and moment of inertia that can make high frequency scanningdifficult or impossible. For example, massive, high-inertia mirrors mayrequire impractically high voltage levels to actuate the mirrors. Suchmirrors may also result in significant frequency limitations that mayrender them inapplicable for certain applications. For example,displacing a mirror having an active area with a diameter of about 5-10mm to about 10-20° with respect to the reference surface may requirelinear displacements of over 1000 μm out of the plane. Under suchconstraints, it may be desirable for a light deflector of a LIDAR systemto exhibit the following characteristics:

-   -   a. Large area/diameter (e.g., in order to enable collection of a        larger amount of light from the scene);    -   b. Large displacement angles (e.g., in order to cover a wide FOV        by a single deflector);    -   c. High operational frequency (e.g. to effectively scan the        wider FOV in short time, or in a greater spatial resolution);    -   d. Low susceptibility to interference frequency (e.g., having        low response to frequencies below 1,000 Hz or so).

Traditional MEMS systems, e.g., two-dimensional MEMS systems having amirror whose surfaces are coplanar with corresponding surfaces of themirror support (e.g., frame 2516 of FIG. 25), may require significanttrade-offs between these characteristics. Such tradeoffs may berequired, for example, as a result of the relatively massive, highinertia mirrors typical of such designs.

The disclosed embodiments may overcome one or more limitationsassociated with traditional MEMS scanning systems. In some cases, thedisclosed MEMS scanning systems may include multilayered structures,i.e., structures constructed from two or more wafers, and fabricationtechniques allowing for significant reductions in mirror mass, mirrormoment of inertia, deflector assembly mass, deflector assembly moment ofinertia, etc. As a result, the disclosed MEMS scanning system may offerthe potential for higher operational frequencies, actuation with lowervoltage levels, higher responsiveness, decreased scan times, etc. ascompared to traditional MEMS scanning systems.

As described in more detail below, multilayered MEMS scanning devicesmay be efficiently and less expensively mass-produced, as the differentwafers may be bonded to one another before the wafers are diced to formthe MEMS device. In some embodiments, one or more of the wafers fromwhich the multilayered MEMS scanning device is made may include one ormore layers, in which case the bonding may be between one or more layersof a first wafer and one or more layers of a second wafer. The term“layer” as used herein refers to a functional layer of a wafer, whichcan include a single material layer or may include a plurality of layersof different materials constructed a single wafer and/or substrate). Forexample, a layer may include separately fabricated layer constituentsthat are assembled into a unitary structure (e.g., a layer may include apackaged assembly including a plurality of sub-layers). In all of theembodiments and processes below, some of the options in which thesystems and methods may be implemented are such in which the term“layer” pertains to a single layer of a wafer made from a singlematerial (e.g., silicon, metal, polysilicon). However, the disclosure isnot limited to such implementations.

Unlike single-wafer devices, multilayer MEMS devices, i.e., MEMS devicesfabricated from two or more wafers, may be more customizable in thateach wafer, or each layer of one of the wafers, may be formed of adifferent material, different thickness, different geometricalstructure, etc., based on the desired mechanical properties for aparticular layer. In some cases, each sub-layer in a packaged layerstack in a MEMS scanning assembly may offer different mechanical and/orelectrical properties that together contribute to an overall set ofproperties associated with the final assembly. Additionally, unliketraditional, single-wafer devices, multilayered MEMS devices may bemanufactured in a range of geometries and configurations, some of whichare impossible or impractical in single wafer devices. For example, amultilayered MEMS device may enable the use of longer actuator armspositioned at least partly behind the mirror and not just to the sidesof the mirror. Such longer actuator arms may give the scanner (e.g., aMEMS mirror) a greater range of motion. The systems and methodsdescribed below may also be implemented in other types of MEMS devicesincluding, for example, pistons, sensors, and the like.

FIGS. 25, 26, 27A, 27B, 27C, and 28 illustrate different views ofseveral exemplary MEMS scanning devices, consistent with disclosedembodiments. Each of the drawings illustrate some aspects which may beimplemented for the making of a MEMS scanning device consistent withdisclosed embodiments. It is noted that the drawings are provided by wayof non-limiting examples only, and aspects illustrated in differentdrawings may be combined in some MEMS scanning devices, and that someMEMS scanning devices which are herein disclosed may include aspects notillustrated by the drawings, or in a different manner than the drawnexamples. The general description in the following paragraphs isfollowed by a more specific description of the different drawings. Whilethe following description is not limited to any of the drawn examples,FIGS. 25 and 27 provide two illustrative examples of multilayered MEMSdevices; In FIG. 25 each of the following three components of the MEMSscanning device is implemented on a wafer of its own: mirror, actuators,and restraining springs; In FIG. 27 the same components are dividedbetween two wafers, and some of the components are implemented on morethan one wafer. For the sake of brevity, the terms “the threecomponents” and “the three components of the MEMS scanning device” wouldrefer to some or all of: one or more mirrors of the MEMS scanningdevice, one or more actuators of the MEMS scanning device, and one ormore restraining springs of the MEMS scanning device. It is noted thatthis usage is not intended to limit the scope of the invention in anyway, and that—as described below—the MEMS scanning device may includemore than these three components, and this components may also beimplemented so as to share wafer parts with one another (e.g., the samepiece of silicon may be used for an actuator and for a restrainingspring).

In some embodiments, a MEMS scanning device, for example, for use in aLIDAR or navigational system, may include a movable MEMS mirror (e.g.,2510, 2708) configured to pivot about at least one axis; one or moreactuators (e.g., 2522, 2712) configured to cause pivoting of the movableMEMS mirror about the at least one axis in at least one first direction;and one or more restraining springs (e.g., 2518, 2706) configured tofacilitate pivoting of the movable MEMS mirror about the at least oneaxis in at least one second direction different from the at least onefirst direction. As used herein, “restraining springs” refers to anycomponent or structure configured to provide a restoring force to theMEMS mirror. In some cases, the restraining springs may limit the motionof the mirror in response to actuation and may restore the mirror to anequilibrium position after actuation (e.g., once an actuating voltagesignal has been discontinued). In some embodiments, the at least onesecond direction is opposite the at least one first direction. That is,the restoring force provided by the one or more restraining springs maybe in a direction opposite to (or substantially opposite to) anactuating force intended to cause at least one displacement of the MEMSmirror. In some embodiments, the MEMS mirror may be part of a LIDARsystem scanning unit. A LIDAR scanning unit may include a deflector, asdescribed throughout the present disclosure, comprising a moveable MEMSmirror configured to pivot about at least one axis. Pivoting along asingle axis, for example, may enable scanning of a LIDAR FOV along asingle horizontal or vertical line. Pivoting along two axes may enablescanning of a LIDAR FOV in both horizontal and vertical directions.

The manufacturing techniques described above may be used in fabricatingone or more of the components, each component being fabricated from atleast one wafer, of the disclosed multilayered MEMS scanning system. Forexample, in some embodiments, any one of the movable MEMS mirror, theone or more actuators, or the one or more restraining springs may beconstructed of at least two differing wafers with mechanical propertiesthat differ from each other, where the at least two differing wafers aredirectly bonded together to form a unified structure. In some cases, twoor more of the movable MEMS mirror, the one or more actuators, and theone or more restraining springs may be constructed of at least twodiffering wafers with mechanical properties that differ from each other,where the at least two differing wafers are directly bonded together toform a unified structure. That is, the two or more wafers in such casestogether include components of at least two of the following componentstypes: mirror, actuator, restraining spring. Optionally, the two or morewafers collectively include components of all of the followingcomponents types: mirror, actuator, restraining spring (that is: atleast one mirror and at least one actuator and at least one restrainingspring).

As mentioned above, the different components—which are in charge of theoperational aspects of mirroring, actuation, and restraining—may beimplemented in many different ways across the two or more wafers of theMEMS scanning device. Referring by way of example to the exemplary MEMSscanning device of FIG. 25, different components of the MEMS scanningdevice (referenced 2500 in this drawings) may be implemented indifferent tiers (each including one or more wafers, or one or morewafer-layers), such that each layer of wafer is used at most for one ofthese components (mirror, actuators, restraining springs). In somecases, the disclosed MEMS scanning devices may include three tiers: amirror tier 2502 (interchangeably “active layer” and “mirror layer”), arestraining tier 2504 (interchangeably “restraining layer”), and anactuation tier 2506 (interchangeably “actuation layer”). Each tier maybe formed of a separate wafer (or wafers) having unique mechanicalproperties. As an example, the mirror tier may be formed from a rigid,lightweight, and relatively thin wafer with a reflective surface orcoating. The restraining tier and the actuation tier may be formed fromthicker, but more flexible wafers. Unlike single-wafer scanning devices,which are restricted to a single wafer thickness, the multilayerconstruction including two or more wafers may allow for selection ofproperties relative to each layer of a wafer (whether that layer is usedfor one of the aforementioned three components or to more than one ofthem). For example, a thin layer used for the mirror of the MEMSscanning device may enable reduced mass of the mirror, allowing the MEMSscanning device to function with higher efficiency, responsiveness,frequency, and a greater range of motion. As discussed further below,other tiers (e.g., the restraining tiers)—and the wafer layers fromwhich those tiers are made—may be fabricated to include structures that,while being formed from a single wafer, may have geometries that furthercontribute to the performance of the scanning system (e.g., restrainingtier structures having hollow shapes, voids, etc.). In such multilayeredMEMS scanning systems, the performance tradeoffs associated withsingle-wafer designs may be mitigated or avoided altogether.

Any or all of the components of the disclosed MEMS scanning device (alsoreferred to as “scanning system” and “scanning device”) may beconstructed from a single wafer or, alternatively, may be fabricatedwith a multilayered structure in which structures may be formedseparately from different wafers or materials and may be bonded togetherduring the fabrication process. For example, in some of the disclosedMEMS scanning systems, at least one of the movable MEMS mirror, theactuators, and/or the restraining springs may be constructed of twodiffering wafers bonded together. For example, the restraining springsmay include a first silicon layer constructed from a first waferdirectly bonded to a second silicon layer constructed from a secondwafer. In the example of FIG. 27 this is demonstrated directly byactuators 2706 a and 2706 c. In the example of FIG. 25 this may beachieved, by way of example, by duplicating wafer 2520, so that tier2506 would include two copies of wafer 2520 which are bonded to eachother. Any other of the other wafers in the example of FIG. 25 may alsobe duplicated.

The multilayered structures of the disclosed embodiments may includevarying degrees of overlap with respect to one another. That is, whenthe MEMS scanning device is assembled, overlap parts of adjacent layersor wafers actually touch each other, while non-overlapping parts of onewafer are positioned next to an opening of a neighboring wafer of thescanning device. In some cases, such overlap may provide regions inwhich one layers may be bonded or joined to the adjacent layer, which isimplemented on a different wafer. The overlap may also offer interfacesthrough which tiers of the structure may interact and/or interfacesthrough which the three components may interact with one another. It isnoted that overlapping parts of different wafers do not have to bebonded to one another, and that overlapping parts of non-neighboringwafers may be separated from one another by an opening, or by anintermediate part of an intermediate wafer. In some embodiments, each ofat least two different wafers may be used to form one or more componentsof the scanning system, and the at least two different wafers mayinclude at least a first overlapping portion and at least a secondnon-overlapping portion. For example, the one or more actuators(possibly a part of an actuation tier 2506) may be made to overlap withthe mirror (possibly a part of a mirror tier 2502) by a certain amount.In some cases, 30% of the one or more actuators may overlap with themirror (e.g., the overlapped portions of the one or more actuators maybe positioned below the mirror and may provide a bonding interface forconnecting the actuators and the mirror, either directly or indirectly;an indirect connection is demonstrated in FIG. 25). In some embodiments,a wafer (or wafers) in which the mirror is implemented may include otherfunctionalities (e.g., restraining springs) or support (e.g., frame). Inanother example, at least some of the actuators may be constructed of afirst wafer, and at least some of the restraining springs may beconstructed of a second wafer different from the first. In someembodiments, at least 25% of the first wafer may overlap with the secondwafer.

In some embodiments, at least one of the wafers used to fabricate aportion of the disclosed MEMS scanning system may be formed of aplurality of layers. For example, a particular wafer may include one ormore of a silicon layer, oxide layer, polycrystalline silicon layer,porous silicon layer, aluminum layer, metal layer, piezoelectric layer,and the like—in any order or configuration. As an example, the mirrormay include a silicon layer (or a layer formed of another material) anda reflective coating on the silicon. In another example, the one or moreactuators may have silicon regions upon which a piezoelectric material(e.g., lead zirconate titanate, hereinafter PZT) has been deposited. Inanother example, the MEMS mirror may be implemented on a first siliconlayer of a silicon-on-insulator wafer, and reinforcement structure ofthe mirror may be implemented on a second silicon layer of the samewafer, separated from the first silicon layer by an insulator layer. Insome embodiments, differing wafers or different layers within thediffering wafers may differ in at least one of: thickness, stiffness,material, crystal direction (e.g., 001, 010, etc.), uniformity (e.g.hole, indentations, surface deformations, etc.), and mechanicalstrength. As previously described, the ability to customize theproperties of each wafer or layer used in the multilayered scanningdevice may provide greater control over the behavior and performance ofthe scanning system.

In some embodiments (e.g., in the example of FIG. 27), the one or moreof the restraining springs may be constructed from multiple wafers andarranged such that at least one restraining spring in the system mayexhibit a multilayered structure. In some cases, a first restrainingspring may be stacked upon at least one other restraining spring. As anexample, a single restraining “spring,” which may refer to a portion ofstructure configured to provide a restoring force to the mirror, may beconstructed from two wafers in a stacked configuration to increase thespring's resistance to movement (e.g., increase a spring constantassociated with the restraining spring/member/structure). In somearrangements, a first restraining spring and a second restraining spring(from among a plurality of restraining springs, for example) maycompletely overlay one another. In another arrangement, the firstrestraining spring and the second restraining spring may only partiallyoverlay one another. In other embodiments, the first restraining springand the second restraining spring may share no overlapping regions.

In some examples, at least one first restraining spring and at least onesecond restraining spring may be bonded together to form a unitarymultilayered spring structure configured to facilitate pivoting of themovable MEMS mirror (e.g., in a direction toward an equilibrium positionof the mirror, in a direction opposite to an actuation direction of themirror, etc.) In some embodiments, the wafers used to form systemcomponents such as the restraining springs may be formed of differentmaterials. These different materials may exhibit different mechanicalproperties, and those mechanical properties may influence theperformance of the system components, such as the restraining springs.For example, strategically arranging restraining springs havingdifferent stiffnesses relative to the mirror may cause the mirror tohave a more restrictive range of motion in one direction (e.g., along anx-axis) than in another direction (e.g., along a y-axis).

In the disclosed MEMS scanning system, certain regions of adjacentwafers may be bonded to one another while other regions may remainunbonded, even if overlapping. Those areas where there is no bonding mayenable movement of a first wafer relative to second wafer. On the otherhand, bonded regions may directly join together two adjacent waferswithout intervening layers such that at least portions of the adjacentwafers physically touch or interact with one another.

In some cases, adjacent wafers may be bonded using adhesives. Forexample, an adhesive may be used to bond adjacent wafers or layerstogether. Any other type of adhesive bonding (interchangeably “gluebonding”) which is used for wafer bonding may also be used. In othercases, adjacent wafers may be bonded without using adhesives. As usedherein, “adhesiveless bonding” may refer to a process in which wafers orlayers are joined together without the use of an adhesive. For example,such adhesiveless bonding may include pressing adjacent wafers togetherunder high pressure. In the disclosed embodiments, wafers or layers maybe bonded together using, for example, direct bonding, surface activatedbonding, plasma activated bonding, anionic bonding, eutectic bonding,etc. By using such bonding processes, wafers may be directly bondedtogether without the use of adhesive. Any of the disclosed bondingprocesses may be automated for efficient mass production of devices.

In some embodiments, the movable MEMS mirror may constitute a either ahinged or hingeless MEMS mirror. The MEMS mirror may be moved or pivotedby a number of actuators. The scanning system may include a number ofmovable MEMS mirrors, each movable MEMS mirror being associated with adifferent group of actuators and/or a different group of restrainingsprings. In some embodiments, the plurality of actuators are configuredto cause pivoting of the movable MEMS mirror in two first directions,and the plurality of restraining springs are configured to facilitatepivoting of the movable MEMS mirror in two second directions, therebythe enabling the movable MEMS mirror to pivot about two distinct axes.For example, in one example a first actuator may enable rotation of amirror in a positive direction about an X axis, and a second actuatormay enable rotation of the mirror in a positive direction about a Yaxis. In this example, a first restraining spring may produce arestoring force that may cause rotation of the mirror in a negativedirection about the X axis. A second restraining spring may produce arestoring force that may cause rotation of the mirror in a negativedirection about the Y axis.

As noted, the movable MEMS mirror may be constructed of a first waferand a plurality of actuators that interact with the mirror may beconstructed (partly or wholly) of one or more second wafers withmechanical properties or characteristics that differ from the mechanicalproperties or characteristics of the first wafer. For example, thethickness of the second wafer may be greater than the thickness of thefirst wafer (or at least, greater than the thickness of a device layeron which the mirror surface is manufactured). For example, the thicknessof the actuators may be greater than the thickness of the mirror. Sucharrangements may enable a significant range of motion produced by theactuators while reducing the mass of the mirror. In another example, thestiffness of the first wafer may be greater than the stiffness of thesecond wafer. In another embodiment, the movable MEMS mirror may beconstructed of a first wafer and one or more of the restraining springsmay be constructed of at least one second wafer with mechanicalproperties that differ from mechanical properties of the first wafer. Inanother embodiment, one or more of the actuators may be constructed of afirst wafer and one or more of a plurality of restraining springs may beconstructed of a second wafer with mechanical properties that differfrom mechanical properties of the first wafer. It will be clear to aperson who is of skill in the art that the examples provided in thisparagraph are not intended to form an exhaustive list, and that otherways of implementing different functions in different wafers may beimplemented in other embodiments of the disclosed scanning MEMS device.

In some embodiments, one or more of the scanning system actuators mayinclude a layer of PZT, or any other piezoelectric material. Forexample, one or more of the actuators may be constructed from a waferonto which a piezoelectric layer is formed. In contrast, the movableMEMS mirror may be constructed of a different wafer, and thepiezoelectric layer may be deposited on yet another separate wafer. Sucha configuration may enable pivoting of the movable MEMS mirror byactivating the piezoelectric material of the one or more actuators. Forexample, the piezoelectric layer on one or more of the actuatorstructures may be activated by applying a voltage to the piezoelectricmaterial, resulting in a mechanical strain on the piezoelectric materialchanging a dimension of the material. As a result, depending on theorientation of the piezoelectric material relative to an actuator arm,an applied voltage to the piezoelectric material can cause movement of amirror actuator in a MEMS scanning system. The actuators of the presentscanning systems, however, may be activated using any other suitabletechniques. For example, other activation methods may includeelectrostatic actuation, electromagnetic actuation, electromechanicalactuation.

In the presently disclosed scanning systems, the actuators may interactwith the scanning mirror through any suitable interaction structure. Insome cases, the material from which one or more actuators are formed maybe bonded or otherwise connected directly to the mirror, and motion ofthe actuators may impart desired motion to the scanning mirror. In somecases, the actuators may be physically connected to a moveable MEMSmirror by interconnects (e.g., flexible interconnects whose flexibilityis much higher than that of the actuators, such as by a factor of atleast ×2, ×5, ×10, etc.). Motion of the actuators may be translated tothe mirror through the interconnects. In some cases, the interconnectsmay form a continuous connection between the actuators and the mirrorand may be formed of a relatively flexible material (e.g., a materialthat may flex in response to movement of the actuators or that mayexhibit a restoring force in the absence of actuation to assist inreturning the mirror to an equilibrium position).

FIG. 25 illustrates an exploded view of an exemplary MEMS scanningdevice 2500, consistent with disclosed embodiments. MEMS device 2500 mayinclude three distinct tiers: a mirror tier 2502, a restraining tier2504, and an actuation tier 2506. In other embodiments, MEMS device 2500may include any number of tiers of various geometries configured in anyorder. For example, the mirror may be implemented on a first wafer, andboth the actuation and restraining of the mirror may be achieved by asecond wafer other than the first wafer. It is noted that any of thevariations, functionalities, structures and so forth discussed belowwith respect to the MEMS scanning device of FIG. 25 may also beimplemented if applicable to any other MEMS scanning device discussed inthe present application, mutatis mutandis, even if not explicitly stated(e.g., for reasons of brevity), and even if the relevant MEMS scanningdevice includes components which are not neatly dividable into separatefunctional tiers. Furthermore, any of the variations, functionalities,structures and so forth discussed below with respect to the MEMSscanning device of any of the following figures may also be implementedif applicable to any other MEMS scanning device discussed in the presentapplication, mutatis mutandis, even if not explicitly stated (e.g., forreasons of brevity).

Mirror tier 2502 may be formed of a wafer 2508 having an active area2510. In some embodiments, a reflective coating may be deposited on thesurface of wafer 2508 to form a flat and continuous surface (e.g., aspecular surface reflective to light). In other embodiments, wafer 2508and mirror surface 2510 may have various degrees of curvature to form aconcave or convex mirror surface. Wafer 2508 and/or active area 2510 mayinclude one or more bonding sites 2512. Bonding sites 2512 may bepositioned on both the top and bottom of the mirror tier 2502, or oneither one of the top or bottom of mirror tier 2502. In someembodiments, mirror tier 2502 may further include one or morereinforcement ribs formed of a rigid material. In some embodiments, thereinforcement ribs may be formed on another wafer bonded to the wafer ofthe mirror, e.g. within optional backing structure 2514 of FIG. 25. Insome embodiments, and as shown in FIG. 25, mirror tier 2502 may beformed of a relatively thin wafer, such that it has a thickness lessthan other wafers used in the fabrication of the scanning device. Insome embodiments, the mirror tier 2502 may be fabricated from a layeredwafer 2508 (e.g., a silicon-on-insulator wafer, SOD in which the activearea 2510 of mirror tier 2502 is implemented on a thin layer (e.g.,silicon) of the wafer, and a supporting substructure (e.g.,reinforcement ribs such as ribs 2722 of FIG. 27B) are implemented on athicker layer of the same wafer. The thin layer may have thickness lessthan other wafers used in the fabrication of the scanning device. Such athin mirror structure may significantly reduce the mass and inertia ofmirror tier 2502 as compared, for example, to two-dimensional scanningdevices made primarily from a single wafer (where the mirror typicallyhas a thickness the same as or similar to the actuators and/orrestraining springs). Manufacturing the mirror tier separate from theother tiers of the scanning device may also enable fabrication of themirror from a material having a lower density than materials used inother tiers, which may also reduce the mass of the mirror relative totraditional designs. Active area 2510 may include a reflective coatingdeposited on the surface of wafer 2508 or a reflective layer bonded towafer 2508.

Restraining tier 2504 may include a backing structure 2514 disposedwithin a frame 2516 and held in place with one or more springs 2518. Insome embodiments, backing structure 2514, frame 2516, and springs 2518may be formed by etching a single wafer and, thus, form a unitarystructure. In some embodiments, backing structure 2514 may be formed ofthe same material and have the same mechanical properties as wafer 2508.In other cases, however, backing structure 2514 may be formed of adifferent material with different mechanical properties as compared tothe material of wafer 2508. In some embodiments, and as described below,frame 2516 may be formed in any geometry such that support 2514 fitswithin the cavity. Backing structure 2514 may be held in the frame 2516by the one or more springs 2518. Springs 2518 may be, for example,metallic springs, wire filaments, an elastomeric material, and/or may beetched from a wafer used to form backing structure 2514 and frame 2516.In some embodiments, all of the springs may exhibit the same or similarspring constants such that they provide the same or similar restoringforces under similar displacements. In other embodiments, one or more ofthe springs may have a different spring constant relative to othersprings in the restraining tier 2504, such that different restoringforces may be provided to the support 2514 (and, therefore, to mirrortier 2502) in response to the same or similar displacement levels.Optionally, the restraining springs and/or actuators may be connecteddirectly to the active area 2510 and not via a support 2514 (which mayor may not be implemented). The restraining springs in such animplementation may be, for example, implemented by the silicon substrateof actuators 2521, since such silicon tends to be restored to itsoriginal unstressed position. In such a case, the relative strength ofsprings 2518 may be significantly lower, possibly serving mostly as aflexible interconnect to the support of the mirror. An example in whichthe restraining springs are implemented together with the actuators isprovided with respect to FIG. 27A.

As noted, in some embodiments, springs 2518 may be relatively thin andmay be etched from the wafer used to provide frame 2516 and form support2514. In other cases, springs 2518 may be thicker than the mirror and/orfrom the actuators. In some cases, controlling the etched thickness ofsprings 2518 may enable selection of a desired spring constant. In othercases, higher spring coefficients may be achieved, for example, bystacking more than one restraining wafer on top of one another, e.g., asdescribed below with reference to FIG. 27A. Restraining elements ofdifferent restraining tiers (if implemented) may be co-located invertical planes (i.e., may partly overlap each other). The number ofrestraining elements (e.g., springs 2518) in each restraining tier maybe the same or different than the number of actuators of the actuationtier. Additionally, the number of restraining elements (e.g., springs)in different restraining tiers may be the same or different from oneanother. Distributing springs (or other restraining elements) amongdifferent tiers may enable the use of thinner restraining tiers (e.g.5-10 μm). The structure and the number of restraining tiers included ina particular scanning system, and the number and structure of associatedsprings may be determined based on operational frequency requirements(e.g. scanning frequency) of the scanning system. In some cases, it maybe desirable to have a certain frequency response profile provided bythe restraining elements. For example, it may be desirable to have a lowresponse to frequencies below 2,000 Hz, 1,000 Hz, 700 Hz or 250 Hz,etc., where vibrations associated with vehicle operation may occur.

Springs 2518 may be configured to provide a restoring force to themirror 2510 (e.g., via support 2514) in response to displacement of themirror 2510 and/or backing structure 2514 caused by forced applied byactuators 2521. For example, springs 2518 may resist the motion impartedto the mirror 2510 by actuators 2521, such that subsequent to anactuation event in which the mirror 2510 is displaced, the restoringforce provided by springs 2518 may pull the displaced mirror into itsoriginal position (e.g., an equilibrium position when no actuators areactivated). In some embodiments, springs 2518 may also mitigate theeffects of low-frequency vibrations on the mirror. For example, in aLIDAR system, springs 2518 may lessen the effects of vibrations causedby vehicle movements, engine operation, road surface irregularities,etc.

Actuation tier 2506 may be formed of a wafer 2520 into which is etchedor otherwise formed one or more controllable actuators 2522 disposed onarms 2521 extending from the frame formed by an opening in wafer 2520.The actuators 2521 may optionally be formed within an opening etched orotherwise formed in wafer 2520. Actuators 2521 may provide force to moveand/or pivot the mirror 2510. In one embodiment, actuator 2521 may bemade wholly or partially of semiconductor (e.g., silicon). On each arm2521 of the actuators may be formed a piezoelectric layer (e.g. PZT,aluminum nitride, etc.), which changes in dimension in response toelectric signals applied by an actuation controller, for example.

In some embodiments, one or more actuators 2521 may be formed wholly orpartially from a piezoelectric material, e.g., PZT, that deforms when avoltage is applied. In some embodiments, wafers 2516 and 2520 mayoverlap in the scanning system assembly by any suitable amount or degreesuch that certain portions of the structures in the restraining tier atleast partially overlap structures in the actuation tier. For example,the wafer 2520 used to form actuation tier 2506 may overlap the wafer2516 of the restraining tier such that at least 25% of the area on asurface of the restraining tier wafer overlaps with an adjacent surfaceof the actuation tier. In other embodiments, any combination of wafer2508, backing structure 2514, frame 2516, and springs 2518, and wafer2520 may partially or completely overlap. In some embodiments, more orfewer actuators than shown in FIG. 25 may be used. Differentconfigurations of actuators, for example, may provide symmetric orantisymmetric actuation. Other actuation techniques known in the art mayalso be used, in addition to or instead of piezoelectric actuation.

As shown in FIG. 25, a PZT layer 2522 may be included on each actuatorarm. In some embodiments the PZT layer 2522 may be disposed on only oneside of the actuator arm. Such a configuration may enable motion of theactuator primarily in one direction (e.g., an upward bending of eachactuator arm). In other cases, a PZT layer may be disposed on both sidesof the actuator arm. Such a configuration may enable movement of eachactuator arm in more than one direction (e.g., an upward bending and adownward bending of each actuator arm).

In some embodiments, the arms 2521 of the actuation tier 2506 mayprovide the restoring force directly to the mirror 2510 (e.g., where theretaining springs 2518 are omitted in favor of using arms 2521 asretaining springs). In such cases, the PZT structures on arms 2521 mayfunction as the actuating elements responsible for pivoting the MEMSmirror in a first direction, and the arms 2521 may be responsible forproviding a restoring force to the MEMS mirror in order to pivot themirror in a direction opposite to the first direction (e.g., after anactuation applied to a PZT structure is discontinued). Referring to themechanical couplings of different elements of MEMS scanning system 2500to one another (e.g., actuators, springs, mirror, frame, support, arms),it is noted that some or all of these couplings may be rigid (e.g.,direct bonding), while some or all of these couplings may be flexible(e.g., connected via flexible interconnects or via flexible parts of therespective components of MEMS scanning system 2500).

In some embodiments, MEMS scanning device 2500 may include additionaltiers and/or components, some or all of which may optionally beimplemented on wafers other than these used for the actuation,restraining and actuation tiers. For example, a transparent wafer may bepositioned in front of the mirror tier for transmission of light to andfrom the mirror. In some embodiments, an additional tier (not shown) mayhold the mirror's controller and/or driving circuitry.

The tiers included in scanning device 2500 may be bonded to one anotherat any suitable location. In some cases, the interlayer bonding may beaccomplished through designated coupling surfaces 2512, for connectingone tier to another tier—either below or above it. While the differenttiers of device 2500 in FIG. 25 are shown as being bonded throughcertain coupling surfaces 2512 (through which the vertical dashed linesextend), any number of coupling surfaces or bonding locations may beused to join adjacent wafers to one another. As previously noted,adjacent wafers may be bonded to one another via any suitable bondingtechnique, such as, for example: direct bonding, surface activatedbonding, plasma activated bonding, anodic bonding, eutectic bonding,pressure sensitive adhesives, bonds formed through intermediate orintervening buffer layers, etc. In some embodiments, bonding techniquesthat may be implemented at relatively low temperatures (e.g., below150°, 200° Celsius) may be used to bond wafers without disrupting theactuation material, e.g., the deposited PZT. With respect to any of thetiers discussed below, each tier (or tier type) may include morecomponents than illustrated and/or discussed, and possibly also to omitsome.

FIGS. 26A-26C are illustrations of a cross-sectional view of anexemplary MEMS scanning device 2600 having a mirror tier 2602, arestraining tier 2604, and an actuation tier 2606. In a nonlimitingexample, the cross-sectional views of FIGS. 26A-26C may be cross sectionof MEMS scanning device 2500 along line A-A shown in FIG. 25. Mirrortier 2602 may be the same as mirror tier 2502 and may include a thinwafer with a reflective coating (or, as mentioned above, a thin devicelayer with a reflective coating). Restraining tier 2604 may be the sameas restraining tier 2504 and may be formed of a single wafer.Restraining tier 2604 may include frame 2616 and backing structure 2614.Actuation tier 2606 may be the same as tier 2506 and may include a wafer2620 with arms 2621 a and 2621 b supporting actuators 2622 a and 2622 b,respectively. tiers 2602, 2604, 2605 may be bonded together at a highpressure to form a single structure 2600. Tiers 2602, 2604, and 2606 maybe implemented on two wafers, on three wafers, or more. In each of thesecases, two or more tiers may be formed on one wafer, as long as two ormore differing wafers are used.

FIGS. 26A-26C show the MEMS scanning device in different actuationstates. For example, FIG. 26A illustrates an equilibrium position ofdevice 2600 where no voltage is applied to any actuators of theactuating layer 2606. As a result, backing structure 2614 and itsassociated mirror tier are in an equilibrium position. FIG. 26Billustrates the state of scanning device 2600 when both actuators 2622 aand 2622 b have a voltage applied. When a voltage is applied toactuators 2622 a and 2622 b located at 180 degrees relative to oneanother, the subsequent upward bending of these opposite side actuatorsresults in displacement of the entire support and mirror upward therebyraising backing component 2614 and mirror tier 2602. FIG. 26C, on theother hand, shows tilting of the backing component and mirror assemblyby actuation of actuator 2622 a differently relative to an opposite sideactuator 2622 b. For example, in some cases, actuator 2622 a may beactuated while actuator 2622 b is left inactive. In such a case, theupward bending of actuator 2622 a and lack of bending of actuator 2622 bwill cause the mirror to tilt. Such tilting of the mirror may also beachieved (where available) by causing one actuator to bend in adirection opposite to another actuator. For example, as shown in FIG.26C, actuator 2622 a has received an applied voltage causing an upwardbend while actuator 2622 b has received an applied voltage causing adownward bend. As a result, the mirror is tilted about an axis normal toand extending through the center of the cross-sectional view of FIG.26C. In another example, the mirror may reach the position shown in FIG.26C from its position shown in FIG. 26B by removing the voltage appliedto actuator 2622 b, thereby allowing the tension of arm 2621 b to pullthe mirror into a tilted position. From the actuation states shown inboth FIGS. 26B and 26C, once the applied voltages are removed, theactuators 2622 a and 2622 b can relax, and the springs of therestraining tier (whether implemented in a separate wafer, as the samearms 2622, or in any other ways) can assist in returning thesupport/mirror to its equilibrium state.

FIG. 27A is an illustration of another exemplary embodiment of amultilayered two-dimensional MEMS scanning device. MEMS device 2700 mayinclude two wafers 2702 and 2704 (including corresponding structuresfabricated from the wafers) bonded together (e.g., using one of thepreviously described direct bonding methods). Wafer 2702 may be formedof a different material than wafer 2704, or be otherwise differenttherefrom. In some cases, wafer 2702 may have at least one mechanicalproperty (e.g., rigidity, flexibility, toughness, tensile strength,etc.) that is different from that of wafer 2704. Wafer 2702 may form aframe and four arms 2706 a, 2706 b, 2706 c, and 2706 d. Each arm may bejoined to the active area/mirror 2708 by a flexible connector 2710. Insome embodiments, the arms and flexible connectors may be formed byetching away portions of a wafer.

In some embodiments, piezoelectric actuators 2712 may be disposed oneach of the four arms. When a voltage is applied to one or more of theactuators 2712, the actuator may contract, causing its respective arm tolift the active area 2708 upward via its respective connector 2710.Vertical translation of the active area/mirror without tilting may beaccomplished by activating all of the actuators such that all displaceequally upward. On the other hand, tilting of the active area/mirror maybe accomplished, for example, through differently actuating opposingarms.

Arms 2706 a, 2706 b, 2706 c, and 2706 d may act as restraining springsby providing a restoring force to active area 2708 via flexibleconnectors 2710. For example, each arm may exhibit a particularstiffness to provide a restoring force to the mirror in the absence ofan actuation signal applied to a corresponding PZT structure/actuatorlocated on the arm.

In some embodiments, arms 2706 a, 2706 b, 2706 c, and 2706 d may beformed of a single wafer or even a single wafer-layer, e.g., wafer 2702.In such an embodiment, each arm may have the same mechanical properties,e.g., the properties of wafer 2702, and may have the same mechanicalproperties of each other.

In another embodiment, one, some or all of arms 2706 a, 2706 b, 2706 c,and 2706 d may be formed of two or more different numbers of wafers. Forexample, as shown in FIG. 27A, two arms, arms 2706 a and 2706 c, may beformed of two wafers 2702 and 2704, while the other arms, 2706 b and2706 d, may be formed of a single wafer, e.g., wafer 2702. Etched wafer2704 may form two arms (arms 2706 a and 2706 c) and may be bonded towafer 2702 using a previously described method, e.g., contact bonding,plasma bonding, and the like.

In embodiments in which the arms have differing numbers of wafers and/ordiffering number of wafer layers, different arms may have differentproperties based on the number of wafers and/or layers and/or mechanicalproperties of the wafers and/or layers forming each of the arms. Forexample, arms 2706 b and 2706 d may have a different range of motionthan arms 2706 a and 2706 c. As an example, if each actuator 2712 exertsthe same amount of force upon activation, arms 2706 a and 2706 c mayexperience less displacement as a result of the additional resistanceprovided by wafer 2704.

The degree of motion of each respective arm may be at least partiallydependent on the materials used to form that arm and also based on thestructure of the arm. For example, arms 2706 a and 2706 c may offer agreater range of motion than arms 2706 b and 2706 d if arms 2706 a and2706 c are made from a more flexible material than arms 2706 b and 2706d. In such embodiments, similar actuation voltages applied to theactuators of arms 2706 a and 2706 b may result in different degrees ofmotion.

In some embodiments, the one or more actuators (whether or notimplemented as an actuation tier) may be constructed of wafer 2702 so asto include arms 2706 a, 2706 b, 2706 b, and 2706 d and actuators 2712.It is noted that the term “actuator” may pertain to only the activeparts of the scanning device (e.g., to the PZT layer), but may also beused to refer to a larger structure which further include a passive partof the MEMS scanning device that moves together with the active part orwhich carries it (e.g., to both the PZT part and the silicon part of therespective arm). Furthermore, in some implementation the active PZT partmay be implemented not as part of a wafer but rather as a stand alonePZT component which is bonded to a wafer (e.g., to a silicon arm of thatwafer).

The one or more restraining springs (whether or not implemented as arestraining tier) may be formed of wafer 2704 and include arms 2706 aand 2706 c. In some embodiments, the one or more restraining springs maybe formed of all the arms. Thus, the wafers forming the one or moreactuators and the one or more restraining springs may includeoverlapping portions and non-overlapping portions. For example,overlapping portions 2706 a and 2706 c and non-overlapping portions 2706b and 2706 d. In some embodiments, at least 25% of the wafer 2702overlaps with wafer 2704. In other embodiments, e.g., device 2500 shownin FIG. 25, the surface area of support 2514 or the surface area of arms2521 may be varied such that the actuation tier 2506 and restrainingtier 2504 overlap to varying degrees.

FIG. 27B illustrates an embodiment of a mirror tier 2714. Mirror tier2714 may include an active layer 2718, e.g., a reflective coating,deposited on a wafer 2720. Wafer 2720 may be etched to include one ormore voids defined by ribs 2722. The voids may be etched into wafer 2720(e.g., at a handle layer of a SOI multilayered wafer) or may be formedas a result of bonding one or more components together to form athree-dimensional structure. Ribs 2722 may provide mechanical strength,stability, and/or durability to mirror tier 2714 without adding excessmass to the tier. Thus, ribs 2722 may reinforce mirror tier 2714 whileadding less mass than a solid reinforcement layer. The number of voidsand/or ribs 2722 may vary depending on the desired mechanical propertiesand/or desired mass of mirror tier 2714. It is noted that the mirrorcomponents of FIG. 27B may also optionally be implemented in a waferwhich includes at least a part of one or more actuator and/or of one ormore restraining springs, and not necessarily as a standalone tier.

FIG. 27C illustrates a wafer which includes the actuation andrestraining components of a MEMS scanning device (i.e., at least oneactuator and at least one restraining spring), collectively referred toas tier 2716. The actuators and restraining springs of FIG. 27C can beconnected, for example, to the mirror tier of FIG. 27B. Tier 2716 mayinclude a wafer 2724, e.g., a silicon wafer, into which a plurality ofarms 2726, flexible connectors 2728, and support area 2730 are etched.The illustrated example includes three arms, which enabletwo-dimensional scanning of the mirror. The lower surface of mirror tier2714 may be bonded to the support area 2730 using one of the previouslydescribed bonding methods. Each arm 2726 may include an actuator 2732,e.g., PZT layer. Each arm 2726 may function as a restraining spring thatprovides a force to return the device to an equilibrium position afteractivation of one or more of the actuators 2732. In addition, each arm2726 may restrict the motion of the mirror tier 2714 by limiting themotion of an activated actuator, e.g., by resisting deformation causedby the contraction of an actuator. Tier 2716 may further includeflexible connectors 2728, which are etched from the wafer 2724 such thatthe mirror tier 2714 is indirectly acted upon by each actuator 2732.

FIG. 28 is an illustration of another embodiment of a one-dimensionalMEMS scanning device, consistent with disclosed embodiments. MEMS device2800 is formed of a mirror 2808 implemented on one wafer, and a secondsingle wafer 2802 which includes both actuators and restraining springs(and therefore may be considered as acting as both the actuation tierand the restraining tier). Mirror 2808 may be formed of a wafer having areflective coating, as previously described. Flexible wafer 2802 may beformed into a frame having one or more arms 2804 a and 2804 b. Each armmay be bonded to piezoelectric actuators 2806 a and 2806 b,respectively, configured to contract when a voltage is applied, therebydeforming the arm on which the actuator is deposited and raising therespective side of mirror 2808. Each arm 2804 a and 2804 b mayadditionally act as a restraining spring by resisting deformation andpulling the mirror 2808 back into its equilibrium position.

In some embodiments, the mirror 2808 may be pivoted to varying degreesalong a central axis by activating each actuator 2806 a and 2806 b incombination or individually, simultaneously or in tandem. In otherembodiments, the movement of the mirror may be modified by, for example,changing the geometry of one or both arms, depositing the piezoelectricmaterial in a different geometry on one or both arms, and/or bonding themirror tier 2808 to a different area of one or both arms. Thus, themirror may pivot to scan a FOV without the use of a hinge.Alternatively, a hinge may be implemented (e.g., as part of the mirrortier 2808, connected between the mirror and its frame (not shown), inorder to restrict the pivoting movement to pivot about the mechanicalhinge.

A MEMS Scanning Device with a Bent Interconnect

FIG. 29 illustrates MEMS scanning device 3000 in accordance withexamples of the presently disclosed subject matter. MEMS scanning device3000 may be used in a LIDAR system (e.g., LIDAR system 100 or any otherscanning LIDAR system), in another electrooptical system (e.g., acamera, a scanning electronic microscope, video projector), or in anyother system.

MEMS scanning device 3000 (hereinafter also MEMS device 3000) mayinclude a movable MEMS mirror 3006 which configured to pivot about atleast one axis (i.e., 1D scanning mirror, 2D scanning mirror). Forexample, MEMS mirror 3006 may rotate with respect to a plane of frame3002 which provides structural stability to MEMS device 3000. In otherimplementations, other types of pivotable surfaces or structures whichare actuated by one or more actuators 3004 to rotate about at least oneaxis may also be implemented instead of a scanning MEMS mirror, mutatismutandis.

MEMS scanning device 3000 include one or more actuators 3004 which areoperable to rotate MEMS mirror 3006 about the at least one axis. In theillustrated example there are four actuators 3004, but any other numberof actuators may also be implemented. The actuators may be piezoelectricactuators, electromechanical actuators, or any other type of actuationknown in the art (e.g., as exemplified above). Each actuator 3004 out ofthe at least one actuator is operable to bend upon actuation to move theMEMS mirror 3006. The actuators 3004 may bend in a directionperpendicular to a plane of the MEMS mirror (if flat) and may also bendin other directions.

The at least one actuator 3004 is connected to the MEMS mirror 3006 byone or more flexible interconnect element 3008 which are connectedbetween the at least one actuator 3004 and the MEMS mirror 3006 andwhich are used for transferring the pulling force generated by thebending of the at least one actuator 3004 to MEMS mirror 3006. It isnoted that in some instances, other mechanical forces (e.g., push,twist) may also be transmitted from the one or more actuators 3004 tothe MEMS mirror 3006 via the one or more flexible interconnects elements3008. Each actuator 3004 may be connected to the mirror by a singleflexible interconnect element 3008 or by one than one flexibleinterconnect elements.

The flexible interconnect elements 3008 may be made from the samematerial and wafer layer (or layers) as the actuators 3004 (or partthereof). For example, the actuators may include a silicon layer body, apiezoelectric element (of a piezoelectric layer of the wafer) that isconfigured to bend the body and move the MEMS mirror when subjected toan electrical field, and metal electrodes for applying the electricalfield to the piezoelectric element. The flexible interconnect in such anexample may be made from the same silicon layer, having the samethickness. Regardless of the exact shape of the flexible interconnectelement 3008, it is significantly narrower (in the plane of themirror/frame) than a width of the actuator 3004 to which it isconnected. For example, the flexible interconnect element 3008 may bemore than 10× thinner than the respective actuator 3004, more than 20×,50×, 75×, 100×, and so on thinner. It is noted that the flexibleinterconnect element 3008 may be connected to the respective actuator atdifferent parts—towards an outer side of the actuator 3004 (e.g., asillustrated for interconnect elements 3008A and 3008B), towards an innerside of the actuator 3004 (closer to MEMS mirror 3006, e.g., asillustrated for interconnect element 3008D), or in an intermediateposition between the inner side and the outer side (e.g., as illustratedfor interconnect element 3008C and an opposing inner side closer to themovable MEMS mirror than the outer side, wherein the first torsionspring is connected to the opposing inner side of the first actuatingarm and the second torsion spring is connected to the outer side of thesecond actuating arm. The way the stresses spread across scanning MEMSdevice 3000 in the different options are obviously different and may beselected for improving the performance of MEMS device 3000 (e.g.,improving frequency of scanning, angle of scanning, and so on).

As exemplified in FIG. 29, each of the one or more flexible interconnectelements 3008 of MEMS device 3000 is an elongated structure whichincludes at least two turns 3010 at opposing directions, each turngreater than 120°. Turns 3010 of a flexible interconnect element 3008are at opposing direction if one of the turns 3010 is at a clockwisedirection and the other turn 3010 is at a counterclockwise direction.

In the illustrated examples, different turns 3010 are at differentangles. For example, some or all of the turns of an interconnect may beat angles greater than 150°, of about 180°, or even at reflex angleswhich are greater than 180°. The turns 3010 may be continuously curvedturns (e.g., as exemplified with respect to the example of interconnectelement 3008B), but may also include one or more angled corners (e.g.,as exampled by interconnect elements 3008A and 3008C). Flexibleinterconnect element 3008D illustrates that different types of turns3010 may be implemented in a single interconnect element 3008. It isnoted that the illustrated MEMS device 3000 includes four differenttypes of flexible interconnect elements 3008 for the sake of exampleonly. In other cases, some or all of the interconnects 3008 of MEMSscanning device 3000 may be identical to each other. Also—while all ofthe actuators 3004 are illustrated as identical and similar to eachother, this is not necessarily the case. It is noted that in addition tothe two or more turns 3010 which are greater than 120°, a flexibleinterconnect element 3008 may also include additional turns. Forexample, a flexible interconnect element 3008 may include one or moreadditional opposing turns of at least 75°.

As exemplified by flexible interconnect elements 3008A and 3008B, aflexible interconnect element 3008 may include a substantially straightportion which is substantially perpendicular to an edge of the MEMSmirror 3006. The straight portion of the interconnect elements 3008 maybe connected directly (as illustrated) or indirectly to the MEMS mirror3006. While not necessarily so, the straight portion may be longer thana bent part of the flexible interconnect element 3008 that includes theat least two turns 3010 (e.g., as illustrated for element 3008A).Optionally, the bent part of the flexible interconnect 3008 has anaccumulated width (e.g., in the radial direction of the illustratedcircular MEMS mirror 3006) that is shorter than that of the elongatedpart (exemplified for interconnect element 3008B by the widths “S” and“B” in the diagram).

The plurality of turns 3010 of the interconnect element 3008 allow it toelongate in the radial direction, and thus a flexibility in the distancebetween the connected actuator 3004 and MEMS mirror 3006 (bothabsolutely, and in the projection on the plane of MEMS mirror 3006). Theplurality of turns 3010 may also allow for dissipation of stresses whichmay be generated if a single 90° turn was implemented instead, forexample. The two or more turns 3010 allow the flexible interconnectelement to “open up” the turns 3010, either in the plane of the MEMSmirror 3006 and/or perpendicular thereto.

Having the bent part of the flexible interconnect 3008 designed with arelatively narrow accumulated width (e.g., shorter than the widths ofthe respective actuator 3004 and/or shower than the width of theelongated part of the same interconnect element 3008) can be used forvarious reasons. For example, such a configuration may enable to havethe bent part (also referred to as “twisted part”) move (e.g., spreadits curls or “open”) primarily in the vertical direction, perpendicularto the plane of MEMS mirror 3006. Thus, instead of creating twistingforces and movement applied to the flexible interconnect element (e.g.,if a single 90° angle is implemented), the bending of the actuator istranslated to a movement which is mostly happening in the verticaldirection. Thus, it uses the energy of the actuator 3004 moreefficiently for the lifting of the part of the MEMS mirror 3006connected to the actuator 3004 by the respective interconnect element3008. A similar configuration may also be used to share the stress moreevenly across different parts of the interconnect elements 3008 and/oracross different parts of the respective actuator 3004, and even toapply the greater stresses during the bending of the actuator 3004 andexpansion/elongation of the interconnect element 3008 towards the MEMSmirror 3006 and the part of the flexible interconnect element 3008closer to the MEMS mirror 3008.

Polygon Deflector

As mentioned above, different types of scanning/movable deflectors maybe used in LIDAR systems (e.g., in LIDAR system 100). One type ofscanning deflectors is scanning polygons which spin (usually in veryhigh angular velocities), whose facets act as mirror and/or prisms whichdeflect light projected onto the facet(s) of the polygon. The polygon isa 3D structure made from a reflective and/or transparent material,having a polygonal intersect. The 3D structure may be a polygonal prism,but this is not necessarily so, and a wide variety of polygonal scannersof non-prism shapes are known in the art. The spinning polygon scannermay be surrounded by a thin medium such as air, nitrogen or vacuum inorder to decrease drag forces on the spinning polygonal scanner.However, in some systems it is useful to immerse the polygon scanner ina fluid, e.g. in order to reduce the effect of mechanical impacts on thedeflector (e.g., resulting from the movement of a moving vehicle). Therefractive index of the liquid 3106 may be selected to resemble (or beequal to) the refractive index of the tank 3104, in order to reduce theoptical effects on light passing between these two media.

FIG. 30A illustrates a reflective scanning polygon 3102 immersed in atank 3104 of liquid 3106. Each facet of the polygon 3102 (or at leastsome of them) reflects light that hit it similarly to a mirror. Sincethe polygon 3102 is rotating, a fixed laser beam impinging upon thepolygon 3102 is scanned over a field of view for each facet. Scanning inthe vertical dimension (if implemented) may be achieved in differentways, such as including an additional scanner for the second axis,implementing non-vertical facets in differing angles, or implementedfacets in different angles one above the other (using one or more lightsources). Some examples of such polygonal scanners are provided withrespect to FIGS. 31A, 31B and 31C. It is noted that beams coming fromdifferent directions (e.g., in the field of view of the LIDAR system)may also be scanned in a similar manner toward a sensor (or anothertarget, whether part of an electrooptical system or not). The reflectionof the light-source beam and that of the returning beam may beimplemented on the same one (or more) of the facets. Examples of somesuch polygons 3102, which may be used in any of the deflectors of FIGS.30A through 30F, are provided in FIGS. 31A, 31B and 31C. FIGS. 30A-30Fillustrate top views illustration of the polygon scanners 3100, andFIGS. 8A-8C include diagonal views of only the polygons 3102, withoutthe tank 3104 in which it is included.

It is noted that different types of liquid (or, more generally, fluids)may be used for immersing the polygon 3102 in the tank. For example, theliquid 3106 may have differing volumetric mass densities, specificweights, viscosity, transparency, refractive indexes, and so on.Likewise, the polygon 3102 may have differing volumetric mass densities,specific weights, viscosity, transparency, reflectivity, and so on.Similarly, the tank 3104 may have differing volumetric mass densities,specific weights, viscosity, transparency, refractive indexes, and soon. The volumetric mass density of the polygon 3102 may be similar tothat of the liquid 3104, e.g. for reducing mechanical forces applied onthe polygon 3102 during rotation, bus this is not necessarily so. Therefractive index of the liquid 3106 may be similar to that of the tank3104, for reducing the number and/or degree of refractions, but this isnot necessarily so.

As demonstrated in FIG. 30A, the incident beam is refracted twice—oncewhen it enters the tank 3104, and the second time when it leaves thetank 3104. The refraction of the beam by the liquid depends on the shapeof the tank 3104. In addition to changing the propagation direction ofthe incident beams, the refractions in the transitions between theoutside air (or other ambient medium, e.g., nitrogen) and the enclosedliquid 3106 may also result in other optical effect such as increase indivergence, aberrations, etc. Furthermore, since the face of the tank3104 is curved and the facets of the polygon 3102 are flat, the liquid3106 (or other fluid) between the components of the embodiment of FIG.30A acts as a lens, which means that the increase in divergence andother optical modifications of the incident beams are not constant indifferent angles (positions) of the polygon 3102.

FIGS. 30B, 30C, 30D, 30E and 30F include several illustrative examplesof polygon scanners 3102 immersed in fluids according to differentaspects of the present invention. The polygon scanners 3102 may be usedin LIDAR system, electrooptical systems (e.g., as exemplified above), orany other optical systems. The fluid may be liquid. The fluid may begaseous. Any one of the polygon 3102 scanners represented by theillustrative examples of FIGS. 30B, 30C, 30D, 30E and 30F may beimplemented in system 100 or in any other types of scanning LIDARsystem. The polygon scanners systems 3100 represented by theillustrative examples of FIGS. 30B, 30C, 30D, 30E and 30F may also beimplemented in any other optical or electro-optical system with anoptical scanning mechanism such as a barcode reader. Any system whichincludes such a polygon 3102 scanner may optionally include one or moreadditional scanning mechanism (e.g. a mirror).

FIG. 30B illustrates a polygon 3102 scanner in accordance with examplesof the presently disclosed subject matter. In the example of FIG. 30B,part of the tank 3104 through which light beams are transferred issubstantially flat, thereby reducing the divergence in the transitionsbetween air and fluid. However, the non-uniformity of the inner part ofthe casing (tank 3104) may interfere with the movement of the rotatingfluid (e.g. resulting in reduced performance, in turbulences, etc.).Non-uniform movement of the rotating fluid may also apply undesiredmechanical forces on the rotating polygon 3102.

FIG. 30C illustrates a polygon 3102 scanner in accordance with examplesof the presently disclosed subject matter. As exemplified in the exampleof FIG. 30C, the polygon 3102 scanner may include a tank 3104 ofnon-uniform width, in which a shape of the exterior part of the tank3104 walls is in different than—and at least partly non-parallel to—ashape of the interior face of the tank 3104 walls. For example, theexterior face of the wall may be substantially flat in at least a partof the circumference of the tank 3104, while the corresponding part ofthe interior face of the walls is curved (e.g., circular, arced orelliptical; possibly in direct continuity of the rest of the interior,as exemplified in the illustration). The part of the exterior of thetank 3104 through which light is transferred to and from the tank 3104may optionally include one or more flat faces (e.g. as exemplified inFIGS. 30C and 30E), but curved faces may also be used (e.g. asexemplified in FIG. 30F). The curving (or flatness) of the exterior facemay be selected based on the refractive indexes of the liquid 3106 andof the tank 3104, and possibly also on the refractive index(is) of oneor more corrective lenses positioned outside the tank 3104 (e.g. asexemplified in FIG. 30D). Such correction lens (or lenses)—ifimplemented—may be directly attached to the tank 3104 (e.g., asillustrated in FIG. 30D), or somewhat remote therefrom. The orientationof the one or more faces of the parts of the tank 3104—exterior and/orcorrective lenses may be determined based on the direction(s) ofincident beam(s) and/or on the direction(s) of beam(s) outbound from thepolygon 3102 (e.g. as exemplified in FIG. 30D).

While the examples of FIGS. 30B, 30C and 30D demonstrate an exteriorwhich is substantially parallel to the interior of the tank 3104 inlarge parts of the tank 3104, this is not necessarily so, and theexterior and interior may have substantially different shapes. Forexample, the interior may be a continuous smooth curve (e.g. circle,ellipse), and the exterior may be an angled shaped (e.g. a polygon 3102,as exemplified in FIG. 30E).

The polygon 3102 scanner may further include other components, such as(but not limited to): a motor, reinforcement structures, locationfeedback mechanism, controller, etc. Any type of motor, reinforcementstructures, location feedback mechanism, controller and/or othercomponents which are known in the art may be used.

Referring to the above examples of polygon scanners systems 3100, it isnoted that the polygon 3102, the tank 3104 (interior and/or exteriorface thereof) and/or the corrective lenses (if any) may be coated inantireflective coatings or other form of optical coatings, many of whichare known in the art.

Optionally, the tank 3104 and/or the corrective lens (if any) mayinclude liquid 3106 (or other fluid) encased in a different casing thanthe liquid 3106 in which the polygon 3102 is at least partly immersed,e.g., in order for having practically the same refractive index.

It is noted that different nuances of the invention where discussed ingreater detail in relation to different drawings. It is noted that anycombination of two or more of the nuances, aspects and features of theinvention discussed above may be implemented in a single polygon scannerassembly 3100.

In view of the non-limiting examples provided above, polygon scannerassembly 3100 may include:

-   -   a. An at least-partly transparent tank 3104;    -   b. An at least-partly transparent fluid 3106 (e.g., liquid),        confined within the tank 3104; and    -   c. A reflective polygon 3102, at least partly immersed in the        fluid 3106, the reflective polygon 3102 operable to reflect an        incidence beam of light arriving from outside the tank 3104 to        provide a deflected beam of light exiting from the tank 3104        outward

A shape of an exterior wall of the tank 3104 is optionally not parallelto a shape of an interior wall of the wall in at least a transferencepart of the wall of the tank 3104 through which at least one of theincidence beam and the deflected beam propagates.

Optionally, the exterior of the transference part includes at least oneflat facet, while the interior of the transference part is a smoothlycurved plane (e.g., as exemplified in FIGS. 30C, 30D, and 30E). It isnoted that the polygon scanner assembly 3100 may include a firsttransference part of the wall for transferring incident beams and asecond transference part of the wall for transferring outbound beams,wherein the first transference part and the second transference partinclude non-parallel faces (e.g., as exemplified in FIG. 30E).

Optionally, a divergence of a light beam scanned by the polygon scanner3102 is substantially unchanged in a plurality of instantaneouspositions of the reflective polygon. Optionally, a divergence of a lightbeam scanned by the polygon scanner 3102 is substantially identical in aplurality of instantaneous positions of the reflective polygon.

In view of the non-limiting examples provided above, polygon scannerassembly 3100 may include:

-   -   a. An at least-partly transparent tank 3104;    -   b. An at least-partly transparent fluid 3106 (e.g., liquid),        confined within the tank 3104;    -   c. A reflective polygon, at least partly immersed in the fluid        3106, the reflective polygon operable to reflect an incidence        beam of light arriving from outside the tank 3104 to provide a        deflected beam of light exiting from the tank 3104 outward; and    -   d. At least one corrective lens for converging or diverging at        least one of the incidence beam and the deflected beam. The at        least one corrective lens may be mechanically coupled to the        tank 3104, such that an outward-facing face of the corrective        lens facing away from the tank 3104 is not parallel to a shape        of an interior wall of the wall in at least a transference part        of the wall of the tank 3104 through which at least one of the        incidence beam and the deflected beam propagates. While not        necessarily so, the at least one corrective lens may be attached        to the exterior of the tank 3104.

Optionally, the fluid 3106 and the tank 3104 have substantially the samerefractive index. Optionally, the fluid 3106 and the corrective lenshave substantially the same refractive index. Optionally, the tank 3104and the corrective lens have substantially the same refractive index.Optionally, the outward-facing face of the at least one corrective lensincludes at least one flat facet, while the interior of the transferencepart is a smoothly curved plane. Optionally, such a polygonal scanningsystem 3100 may include one or more first corrective lens fortransferring incident beams and one or more second corrective lens fortransferring outbound beams, such that the one or more first correctivelens and the one or more second corrective lens include non-parallelfaces. Optionally, a divergence of a light beam scanned by the polygonscanner is substantially unchanged in a plurality of instantaneouspositions of the reflective polygon. Optionally, a divergence of a lightbeam scanned by the polygon scanner is substantially identical in aplurality of instantaneous positions of the reflective polygon.

As aforementioned, a polygonal scanner system 3100 as disclosed abovemay be incorporated into a LIDAR system or any other optical and/orelectrooptical system. such a system may further include othercomponents of a LIDAR system, e.g., as discussed above with respect toLIDAR system 100. For example, such an electrooptical system mayinclude: a light source, operable to emit at least one light beam;polygon scanner assembly 3100 operable to deflect the at least one lightbeam toward a FOV of the electrooptical system, such that the polygonscanner deflects the at least one light beam in different directionswhile being positioned in different instantaneous positions of thepolygon scanner; and a controller for controlling rotation of thepolygon scanner. Such a LIDAR system may also include a sensor fordetecting reflected light of the at least one light beam reflected backfrom one or more objects in the field of view, and a processor operableto process detection information of the sensor, to provide distanceinformation for the one or more objects.

1-93. (canceled)
 94. A LIDAR system, comprising: at least one processorconfigured to: control at least one LIDAR light source for illuminatinga field of view; receive from a group of light detectors a plurality ofinput signals indicative of reflections of light from objects in thefield of view; process a first subset of the input signals associatedwith a first region of the field of view to detect a first object in thefirst region, wherein processing the first subset is performedindividually on each input signal of the first subset of the inputsignals; process a second subset of the input signals associated with asecond region of the field of view to detect at least one second objectin the second region, wherein the at least one second object is locatedat a greater distance from the at least one light source than the firstobject and wherein processing of the second subset includes processingtogether input signals of the second subset; and output informationindicative of a distance to the first object and information indicativeof a distance to the at least one second object.
 95. The LIDAR system ofclaim 94, wherein the at least one processor is further configured toreceive the first subset of the input signals and second subset of theinput signals in a time frame associated with single scanning cycle. 96.The LIDAR system of claim 94, wherein the first region of the field ofview is associated with a foreground portion of the field of view andthe second region of the field of view is associated with a backgroundportion of the field of view.
 97. The LIDAR system of claim 94, whereinthe first region of the field of view and the second region of the fieldof view are spatially separated.
 98. The LIDAR system of claim 97,wherein the at least one processor is further configured to control theat least one LIDAR light source to illuminate the first region of thefield of view in a first illumination level and to illuminate the secondregion of the field of view in a second illumination level lower thanthe first illumination level.
 99. The LIDAR system of claim 94, whereinthe first region of the field of view and the second region of the fieldof view at least partially overlap.
 100. The LIDAR system of claim 94,wherein a same group of detectors generate the first subset of the inputsignals and second subset of the input signals.
 101. The LIDAR system ofclaim 94, wherein a size of at least one of the first region and thesecond region is predefined.
 102. The LIDAR system of claim 94, whereinthe at least one processor is further configured to dynamically define asize of the second region.
 103. The LIDAR system of claim 102, whereinthe at least one processor is further configured to dynamically definethe size of second region based on at least one of ambient lightingconditions, object reflectivity, noise levels, vehicle speed, anddriving environment.
 104. The LIDAR system of claim 94, wherein the atleast one processor is further configured to initially process thesecond subset of input signals by analyzing the input signals of thesecond subset individually, and when no object is definitively detected,process the second subset by combining at least two input signals of thesecond subset.
 105. The LIDAR system of claim 94, wherein the at leastone processor is further configured to initially determine that at leastsome of the second subset of the input signals are associated withinsufficient detection information, and thereafter process the secondsubset by combining at least two input signals of the second subset.106. The LIDAR system of claim 94, wherein the at least one processor isconfigured to cause illumination of the field of view by moving at leastone light deflector in a scanning cycle to deflect light from the atleast one light source such that during the scanning cycle the at leastone light deflector is moved through a plurality of differentinstantaneous positions.
 107. The LIDAR system of claim 106, wherein foreach particular instantaneous position of the at least one lightdeflector, a sensor including the group of light detectors is configuredto generate a portion of the plurality of input signals, each of theinput signals being associated with an output of a corresponding pixelof the sensor.
 108. The LIDAR system of claim 106, wherein first andsecond subsets of the input signals are received during the scanningcycle.
 109. A vehicle, comprising: a body; and at least one processorwithin the body and configured to: control at least one light source forilluminating a field of view; receive from a group of detectors aplurality of input signals indicative of reflections of light fromobjects in the field of view; process a first subset of the inputsignals associated with a first region of the field of view to detect afirst object in the first region, wherein processing the first subset isperformed individually on each of the first subset of the input signals;process a second subset of the input signals associated with a secondregion of the field of view to detect at least one second object in thesecond region, wherein the at least one second object is located at agreater distance from the at least one light source than the firstobject and wherein processing of the second subset includes combininginput signals of the second subset; and output information associatedwith a distance to the first object and information associated with adistance to the at least one second object.
 110. The vehicle of claim109, wherein the at least one processor is further configured to processthe second subset by: applying a first processing scheme on the inputsignals of the second subset to identify an existence of a second objectin the second region; and applying a second processing scheme on theinput signals of the second subset to identify an existence of anadditional second object in the second region not identifiable in thefirst processing scheme.
 111. The vehicle of claim 110, wherein thefirst processing scheme and the second processing scheme includesprocessing together input signals of the second subset associated withadjacent detectors.
 112. The vehicle of claim 110, wherein the at leastone processor is further configured to combine data obtained from thefirst processing scheme and data obtained from the second processingscheme to identify an existence of a third object in the second regionnot identifiable in both the first and the second processing schemes.113. The vehicle of claim 110, wherein the at least one processor isfurther configured to select the first processing scheme based on anillumination scheme associated with the second region.
 114. The vehicleof claim 109, wherein the at least one processor is further configuredto process a third subset of the input signals associated with a thirdregion of the field of view to detect a third object in the thirdregion, wherein the third object is located at a greater distance fromthe light source than the at least one second object and whereinprocessing of the third subset includes combining input signals of thethird subset, and wherein the outputted information is furtherassociated with a distance to the third object.
 115. The vehicle ofclaim 109, wherein the at least one processor is further configured toprocess a third subset of the input signals associated with a thirdregion of the field of view to detect a third object in the thirdregion, wherein the third object is less reflective than the at leastone second object and wherein processing of the third subset includescombining input signals of the third subset, and wherein the outputtedinformation is further associated with a distance to the third object.116. The vehicle of claim 109, wherein the output information furtherincludes information associated with at least one of the followingmeasurements: a velocity of the first object, a surface angle of thefirst object, a reflectivity level of the first object, ambient lightassociated with the first object, ambient light associated with thesecond object, and a confidence level in each of the above measurements.117. The vehicle of claim 109, wherein the at least one processor isfurther configured to control at least one light deflector such thatduring a scanning cycle of the field of view, the at least one lightdeflector is move through a plurality of different instantaneouspositions.
 118. The vehicle of claim 117, wherein the at least oneprocessor is further configured to coordinate the at least one lightdeflector and the at least one light source such that when the at leastone light deflector is located at a particular instantaneous position, aportion of a light beam is deflected by the at least one light deflectorfrom the at least one light source towards an object in the field ofview, and reflections of the portion of the light beam from the objectare deflected by the at least one light deflector toward at least onesensor associated with the group of detectors.
 119. The vehicle of claim109, wherein a representation of the field of view associated with aplurality of pixels is constructible from the plurality of inputsignals, and wherein the at least one processor is further configured tooutput information associated with a first direction to the first objectand a second direction to the at least one second object, the firstdirection being associated with a center of a pixel and the seconddirection being associated with an edge of a pixel.
 120. A method forusing a LIDAR system to determine distances to objects in a field ofview, the method comprising: controlling at least one light source forilluminating the field of view; receiving from a group of detectors aplurality of input signals indicative of reflections of light fromobjects in the field of view; processing a first subset of the inputsignals associated with a first region of the field of view to detect afirst object in the first region, wherein processing the first subset isperformed individually on each of the first subset of the input signals;processing a second subset of the input signals associated with a secondregion of the field of view to detect at least one second object in thesecond region, wherein the at least one second object is located at agreater distance from the light source than the first object and whereinprocessing of the second subset includes combining input signals of thesecond subset; and outputting information associated with a distance tothe first object and information associated with a distance to the atleast one second object. 121-180. (canceled)